Haptic feedback using a field of view

ABSTRACT

One illustrative system disclosed herein includes a computing device in communication with a display device and a sensor. The display device is configured to display a plurality of content and the sensor is configured to detect a field of view of a user of the computing device relative to the display device. The sensor can transmit a signal associated with the field of view to a processor in communication with the sensor. The processor is configured to determine a direction of the field of view of the user based on the signal. The processor is also configured to determine that a content displayed by the display device and associated with a haptic effect is within the field of view of the user. The processor is also configured to determine a haptic effect associated with the content and transmit a haptic signal associated with the haptic effect. The illustrative system also includes a haptic output device configured to receive the haptic signal and output the haptic effect.

CROSS-REFERENCE

This Application is a continuation of and claims the benefit of U.S.patent application Ser. No. 15/390,868, filed on Dec. 27, 2016 andentitled “Haptic Feedback Using a Field of View,” the entire contents ofwhich are incorporated by reference herein for all purposes.

FIELD OF INVENTION

The present disclosure relates generally to user interface devices. Morespecifically, but not by way of limitation, this disclosure relates tohaptic feedback using a field of view.

BACKGROUND

Display devices can be used to provide various content (e.g., images,vides, etc.) to a user of a display device. The display device can alsobe used to create a virtual reality environment that can simulate theuser's physical presence and environment and allow the user to interactwith the virtual objects in the simulated environment. The user may lookat or interact with content or virtual objects displayed via the displaydevices (e.g., by touching or looking in the direction of the content orthe virtual objects). Some display devices, however, may lack hapticfeedback capabilities. Moreover, some display devices and virtualreality devices may not provide haptic feedback relevant to the contentor virtual objects displayed via the display devices or relevant to theuser's interaction with the content or virtual objects.

SUMMARY

Various embodiments of the present disclosure provide systems andmethods for haptic feedback using a field of view.

In one embodiment, a system of the present disclosure may comprise acomputing device communicatively coupled to a display device. Thedisplay device can be configured to display a plurality of content. Thesystem also comprises a sensor communicatively coupled to the computingdevice and configured to detect a field of view of a user of thecomputing device relative to the display device and transmit a signalassociated with the field of view. The system also comprises a processorcommunicatively coupled to the sensor for receiving the signal. Theprocessor can be configured to determine a direction of the field ofview of the user based at least in part on the signal. The processor canalso be configured to determine that a content displayed via the displaydevice and within the direction of the field of view of the user isassociated with a haptic effect. The processor can further be configuredto determine the haptic effect associated with the content and transmita haptic signal associated with the haptic effect associated with thecontent. The system may further comprise a haptic output deviceconfigured to receive the haptic signal and output the haptic effect.

In another embodiment, a method of the present disclosure may comprise:detecting, by a sensor, a field of view of a user of a computing devicerelative to a display device communicatively coupled to the computingdevice, the display device configured to display a plurality of content;transmitting, by the sensor, a sensor signal associated with the fieldof view to a processor; determining, by the processor, a direction ofthe field of view based at least in part on the sensor signal;determining, by the processor, that a content displayed via the displaydevice is within the field of view of the user based on the direction ofthe field of view; determining, by the processor, a haptic effectassociated with the content; transmitting, by the processor, a hapticsignal associated with the haptic effect to a haptic output device; andoutputting, by the haptic output device, the haptic effect.

These illustrative embodiments are mentioned not to limit or define thelimits of the present subject matter, but to provide examples to aidunderstanding thereof. Illustrative embodiments are discussed in theDetailed Description, and further description is provided there.Advantages offered by various embodiments may be further understood byexamining this specification and/or by practicing one or moreembodiments of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

A full and enabling disclosure is set forth more particularly in theremainder of the specification. The specification makes reference to thefollowing appended figures.

FIG. 1 is a block diagram showing a system for haptic feedback using afield of view of view according to one embodiment.

FIG. 2 shows an embodiment of a system for haptic feedback using a fieldof view according to one embodiment.

FIG. 3 shows another embodiment of a system for haptic feedback using afield of view according to another embodiment.

FIG. 4 shows an embodiment of a system for haptic feedback using fieldof view according to another embodiment.

FIG. 5 is a flow chart of steps for performing a method for providinghaptic feedback using a field of view according to one embodiment.

FIG. 6 is a flow chart of steps for performing another method forproviding haptic feedback using a field of view according to oneembodiment.

FIG. 7 is a flow chart of steps for performing another method forproviding haptic feedback using a field of view according to oneembodiment.

DETAILED DESCRIPTION

Reference now will be made in detail to various and alternativeillustrative embodiments and to the accompanying drawings. Each exampleis provided by way of explanation and not as a limitation. It will beapparent to those skilled in the art that modifications and variationscan be made. For instance, features illustrated or described as part ofone embodiment may be used in another embodiment to yield a stillfurther embodiment. Thus, it is intended that this disclosure includemodifications and variations that come within the scope of the appendedclaims and their equivalents.

Illustrative Examples of Haptic Feedback Using a Field of View

One illustrative embodiment of the present disclosure comprises adisplay device, such as a video screen, a computer, or a virtual realityheadset. The display device comprises a sensor, a memory, and aprocessor in communication with each of these elements.

In the illustrative embodiment, the sensor may detect a field of view ora direction of an eye gaze of a user of the display device (e.g., a userlooking at or toward the display device). For example, the displaydevice may comprise a video screen with an embedded or proximate sensorfor detecting the user's field of view with respect to content (e.g.,texts, images, sounds, videos, characters, virtual objects, virtualanimations, etc.) on the video screen. The sensor can transmit a signalto the processor, which determines whether content displayed on thevideo screen and associated with a haptic effect is within the field ofview of the user.

In the illustrative embodiment, the processor can transmit a hapticsignal associated with the haptic effect to a haptic output deviceassociated with the user (e.g., to a smartwatch worn by the user thatincludes the haptic output device) in response to determining that thecontent associated with the haptic effect is within the field of view ofthe user. The haptic output device is configured to receive the hapticsignal from the processor and output one or more haptic effectsassociated with the content in the field of view of the user. In theillustrative embodiment, the haptic output effects can correspond to oneor more events relevant to the content in the field of view of the user(e.g., an interaction, action, collision, or other event associated withthe content). Thus, for example, the user can perceive haptic effectsassociated with a particular virtual character that the user is lookingat.

In the illustrative embodiment, the sensor may also detect a distancebetween the display device and the user and transmit a sensor signalcorresponding to the distance to the processor, which determines acharacteristic of the haptic effect based at least in part on thedistance between the user and the display device. For example, theprocessor may determine a magnitude, duration, frequency, etc. of thehaptic effect based on the distance. In this illustrative embodiment,the processor can modify or adjust one or more characteristics of thehaptic effect based on the distance between the user and the computingdevice and transmit the haptic signal to the haptic output device, whichoutputs the haptic effect. Thus, for example, the user may perceive astronger haptic effect when the user is close to the display device or aweaker haptic effect when the user is far from the display device.

In the illustrative embodiment, the processor may also determine acharacteristic of the haptic effect based at least in part on a virtualor perceived distance between the user and content within the field ofview of the user. For example, a size, location, angle, or othercharacteristic of content displayed via the display device may cause(e.g., by the processor) the content to be perceived by the user asbeing close to the user or far from the user. As an example, a smallvirtual character in a scene displayed via the display device may beperceived by the user as being far from the user. In the illustrativeembodiment, the processor can determine a magnitude, duration,frequency, etc. of the haptic effect based on the virtual or perceiveddistance. As an example, the processor determines a strong haptic effectfor content within the field of view of the user that the user perceivesas being close to the user.

These illustrative examples are given to introduce the reader to thegeneral subject matter discussed here and are not intended to limit thescope of the disclosed concepts. The following sections describe variousadditional features and examples with reference to the drawings in whichlike numerals indicate like elements, and directional descriptions areused to describe the illustrative examples but, like the illustrativeexamples, should not be used to limit the present disclosure.

Illustrative Systems for Haptic Feedback Based on a Field of View

FIG. 1 is a block diagram showing a system 100 for haptic feedback basedon a field of view according to one embodiment. In the embodimentdepicted in FIG. 1, the system 100 comprises a computing device 101having a processor 102 in communication with other hardware via a bus106. The computing device 101 may comprise, for example, a mobile device(e.g., a smartphone), tablet, e-reader, smartwatch, a head-mounteddisplay, glasses, a wearable device, a virtual reality computing device,etc.

A memory 104, which can comprise any suitable tangible (andnon-transitory) computer-readable medium such as RAM, ROM, EEPROM, orthe like, embodies program components that configure operation of thecomputing device 101. In the embodiment shown, computing device 101further includes one or more network interface devices 110, input/output(I/O) interface components 112, and storage 114.

Network interface device 110 can represent one or more of any componentsthat facilitate a network connection. Examples include, but are notlimited to, wired interfaces such as Ethernet, USB, IEEE 1394, and/orwireless interfaces such as IEEE 802.11, Bluetooth, or radio interfacesfor accessing cellular telephone networks (e.g., transceiver/antenna foraccessing a CDMA, GSM, UMTS, or other mobile communications network).

I/O components 112 may be used to facilitate wired or wirelessconnection to devices such as one or more displays 134, gamecontrollers, keyboards, mice, joysticks, cameras, buttons, speakers,microphones and/or other hardware used to input or output data. Storage114 represents nonvolatile storage such as magnetic, optical, or otherstorage media included in computing device 101 or coupled to theprocessor 102.

In some embodiments, the computing device 101 includes a touch sensitivesurface 116. Touch sensitive surface 116 represents any surface that isconfigured to sense tactile input of a user. One or more touch sensors108 are configured to detect a touch in a touch area (e.g., when anobject contacts the touch sensitive surface 116) and transmit signalsassociated with the touch to the processor 102. Any suitable number,type, or arrangement of touch sensor 108 can be used. For example,resistive and/or capacitive sensors may be embedded in touch sensitivesurface 116 and used to determine the location of a touch and otherinformation, such as pressure, speed, and/or direction.

The touch sensor 108 can additionally or alternatively comprise othertypes of sensors. For example, optical sensors with a view of the touchsensitive surface 116 may be used to determine the touch position. Asanother example, the touch sensor 108 may comprise a LED (Light EmittingDiode) finger detector mounted on the side of a display. In someembodiments, touch sensor 108 may be configured to detect multipleaspects of the user interaction. For example, touch sensor 108 maydetect the speed, pressure, and direction of a user interaction, andincorporate this information into the signal transmitted to theprocessor 102. Thus, for example, the computing device 101 can be asmartphone that includes the touch sensitive surface 116 (e.g., a touchsensitive screen) and touch sensor 108 for detecting user input when auser of the smartphone touches the touch sensitive surface 116.

In some embodiments, the computing device 101 comprises a touch-enableddisplay that combines a touch sensitive surface 116 and a display 134 ofthe computing device 101. The touch sensitive surface 116 may beoverlaid on the display 134, may be the display 134 exterior, or may beone or more layers of material above components of the display 134. Inother embodiments, the computing device 101 may display a graphical userinterface (“GUI”) that includes one or more virtual user interfacecomponents (e.g., buttons) on the touch-enabled display and the touchsensitive surface 116 can allow interaction with the virtual userinterface components.

In some embodiments, computing device 101 comprises a camera 130.Although the camera 130 is depicted in FIG. 1 as being internal to thecomputing device 101, in some embodiments, the camera 130 may beexternal to and in communication with the computing device 101. As anexample, the camera 130 may be external to and in communication with thecomputing device 101 via wired interfaces such as, for example,Ethernet, USB, IEEE 1394, and/or wireless interfaces such as IEEE1802.11, Bluetooth, or radio interfaces.

In some embodiments, the computing device 101 comprises one or moresensors 132. In some embodiments, the sensor 132 may comprise, forexample, gyroscope, an accelerometer, a global positioning system (GPS)unit, a range sensor, a depth sensor, a Bluetooth device, a camera, aninfrared sensor, a quick response (QR) code sensor, etc. In someembodiments, the sensor 132 is external to the computing device 101 andin wired or wireless communication with the computing device 101.

In some embodiments, the computing device 101 may comprise a wearabledevice (e.g., glasses, a headset, etc.) and the sensor 132 may compriseany device for detecting eye gaze, line-of-sight, or field of view of auser of the computing device 101. For example, the sensor 132 can detecta direction of the user's field of view with respect to content (e.g.,texts, images, sounds, videos, characters, virtual objects, virtualanimations, etc.) displayed on display 134 or another display device(e.g., a display device 136).

As an example, the sensor 132 may include a camera or be incorporatedinto camera 130. Such a sensor can be configured to capture an image ofthe eye of the user of the computing device 101, and the processor 102can determine the direction of the field of view of the user of thecomputing device 101 relative to content on display 134 or displaydevice 136 based at least in part on the image by using various imageprocessing methods and techniques. In another embodiment, the sensor 132is configured to monitor movements of an eye of the user or muscles nearan eye of the user of the computing device 101 and the processor 102 isconfigured to determine the direction of the user's field of viewrelative to content on the display 134 or display device 136 based atleast in part on the monitored movements. In still another embodiment,the sensor 132 may be configured to monitor or measure electricalactivity of muscles moving the eye of the user of the computing device101 and the processor 102 can be configured to determine the directionof the user's field of view relative to content on the display 134 ordisplay device 136. In some embodiments, the sensor 132 may includeother sensors used to determine a user's intent or volition, including,for example, sensors associated with functional magnetic resonanceimaging (“fMRI”) or electroencephalogram (“EEG”). In still anotherembodiment, the sensor 132 may detect the user's eye gaze, line-ofsight, or field of view through various methods and techniques,including, for example, analyzing the user's body or head posture. As anexample, the sensor 132 can include a head-mounted display or ahead-mounted sensor for detecting a motion of the user's head or fordetecting the user's head posture and transmitting data about the motionof the user's head or data about the user's head posture to theprocessor 102, which can determine the direction of the field of view ofthe user of the computing device 101 based on the data.

In some embodiments, the sensor 132 may detect a location of the displaydevice 136. As an example, the sensor 132 may be a Bluetooth device orother network device configured to detect a location of anotherBluetooth display device (e.g., display device 136 can be a Bluetoothdisplay device) by analyzing signal strength between the sensor 132 andthe Bluetooth display device. In some embodiments, the sensor 132 maydetect a distance between the sensor 132, the computing device 101, or auser of the computing device 101, and the display device 136 (e.g.,based on the strength of the Bluetooth signal between the sensor 132 andthe display device 136). In some embodiments, the sensor 132 may detecta location of the display device 136 or a distance between the sensor132, the computing device 101, or a user of the computing device 101,and the display device 136 via any suitable method or technique.

In some embodiments, the processor 102 may be in communication with asingle sensor 132 and, in other embodiments, the processor 102 may be incommunication with a plurality of sensors 132, for example, a camera, aninfrared sensor, and a Bluetooth device. The sensor 132 is configured totransmit sensor signals to the processor 102.

In some embodiments, the system 100 further includes haptic outputdevice 118 in communication with the processor 102. Haptic output device118 is configured to output a haptic effect in response to a hapticsignal. For example, the haptic output device 118 can output a hapticeffect in response to a haptic signal from the processor 102. In someembodiments, haptic output device 118 is configured to output a hapticeffect comprising, for example, a vibration, a squeeze, a poke, a changein a perceived coefficient of friction, a simulated texture, a strokingsensation, an electro-tactile effect, a surface deformation (e.g., adeformation of a surface associated with the computing device 101),and/or a puff of a solid, liquid, or gas. Further, some haptic effectsmay use multiple haptic output devices 118 of the same or differenttypes in sequence and/or in concert.

Although a single haptic output device 118 is shown in FIG. 1, someembodiments may use multiple haptic output devices 118 of the same ordifferent type to produce haptic effects. Further, in some embodiments,the haptic output device 118 is in communication with the processor 102and internal to the computing device 101. In other embodiments, thehaptic output device 118 is external to the computing device 101 and incommunication with the computing device 101 (e.g., via wired interfacessuch as Ethernet, USB, IEEE 1394, and/or wireless interfaces such asIEEE 802.11, Bluetooth, or radio interfaces). For example, the hapticoutput device 118 may be associated with (e.g., coupled to) a wearabledevice (e.g., a wristband, bracelet, hat, headset, headband, etc.) andconfigured to receive haptic signals from the processor 102.

In some embodiments, the haptic output device 118 is configured tooutput a haptic effect comprising a vibration. The haptic output device118 may comprise, for example, one or more of a piezoelectric actuator,an electric motor, an electro-magnetic actuator, a voice coil, a shapememory alloy, an electro-active polymer, a solenoid, an eccentricrotating mass motor (ERM), or a linear resonant actuator (LRA).

In some embodiments, the haptic output device 118 is configured tooutput a haptic effect modulating the perceived coefficient of frictionof a surface associated with the haptic output device 118. In oneembodiment, the haptic output device 118 comprises an ultrasonicactuator. An ultrasonic actuator may vibrate at an ultrasonic frequency,for example 20 kHz, increasing or reducing the perceived coefficient offriction of the surface associated with the haptic output device 118. Insome embodiments, the ultrasonic actuator may comprise a piezo-electricmaterial.

In some embodiments, the haptic output device 118 uses electrostaticattraction, for example by use of an electrostatic actuator, to output ahaptic effect. The haptic effect may comprise a simulated texture, asimulated vibration, a stroking sensation, or a perceived change in acoefficient of friction on a surface associated with computing device101 (e.g., the touch sensitive surface 116). In some embodiments, theelectrostatic actuator may comprise a conducting layer and an insulatinglayer. The conducting layer may be any semiconductor or other conductivematerial, such as copper, aluminum, gold, or silver. The insulatinglayer may be glass, plastic, polymer, or any other insulating material.Furthermore, the processor 102 may operate the electrostatic actuator byapplying an electric signal, for example an AC signal, to the conductinglayer. In some embodiments, a high-voltage amplifier may generate the ACsignal. The electric signal may generate a capacitive coupling betweenthe conducting layer and an object (e.g., a user's finger or other bodypart, or a stylus) near or touching the touch sensitive surface 116.Varying the levels of attraction between the object and the conductinglayer can vary the haptic effect perceived by a user.

In some embodiments, the haptic output device 118 comprises adeformation device configured to output a deformation haptic effect. Thedeformation haptic effect may comprise raising or lowering portions of asurface associated with the computing device 101. For example, thedeformation haptic effect may comprise raising portions of the touchsensitive surface 116. In some embodiments, the deformation hapticeffect may comprise bending, folding, rolling, twisting, squeezing,flexing, changing the shape of, or otherwise deforming a surfaceassociated with the computing device 101. For example, the deformationhaptic effect may apply a force on the computing device 101 or a surfaceassociated with the computing device 101 (e.g., the touch sensitivesurface 116), causing it to bend, fold, roll, twist, squeeze, flex,change shape, or otherwise deform.

In some embodiments, the haptic output device 118 comprises fluidconfigured for outputting a deformation haptic effect (e.g., for bendingor deforming a surface associated with the computing device 101). Forexample, the fluid may comprise a smart gel. A smart gel comprises afluid with mechanical or structural properties that change in responseto a stimulus or stimuli (e.g., an electric field, a magnetic field,temperature, ultraviolet light, shaking, or a pH variation). Forinstance, in response to a stimulus, a smart gel may change instiffness, volume, transparency, and/or color. In some embodiments,stiffness may comprise the resistance of a surface associated with thecomputing device 101 (e.g., the touch sensitive surface 116) againstdeformation. In some embodiments, one or more wires may be embedded inor coupled to the smart gel. As current runs through the wires, heat isemitted, causing the smart gel to expand or contract, which may causethe computing device 101 or a surface associated with the computingdevice 101 to deform.

As another example, the fluid may comprise a rheological (e.g., amagneto-rheological or electro-rheological) fluid. A rheological fluidcomprises metal particles (e.g., iron particles) suspended in a fluid(e.g., oil or water). In response to an electric or magnetic field, theorder of the molecules in the fluid may realign, changing the overalldamping and/or viscosity of the fluid. This may cause the computingdevice 101 or a surface associated with the computing device 101 todeform.

In some embodiments, the haptic output device 118 comprises a mechanicaldeformation device. For example, in some embodiments, the haptic outputdevice 118 may comprise an actuator coupled to an arm that rotates adeformation component. The deformation component may comprise, forexample, an oval, starburst, or corrugated shape. The deformationcomponent may be configured to move a surface associated with thecomputing device 101 at some rotation angles, but not others. Theactuator may comprise a piezo-electric actuator, rotating/linearactuator, solenoid, an electroactive polymer actuator, macro fibercomposite (MFC) actuator, shape memory alloy (SMA) actuator, and/orother actuator. As the actuator rotates the deformation component, thedeformation component may move the surface, causing it to deform. Insuch an embodiment, the deformation component may begin in a position inwhich the surface is flat. In response to receiving a signal fromprocessor 102, the actuator may rotate the deformation component.Rotating the deformation component may cause one or more portions of thesurface to raise or lower. The deformation component may, in someembodiments, remain in this rotated state until the processor 102signals the actuator to rotate the deformation component back to itsoriginal position.

Further, other techniques or methods can be used to deform a surfaceassociated with the computing device 101. For example, the haptic outputdevice 118 may comprise a flexible surface layer configured to deformits surface or vary its texture based upon contact from a surfacereconfigurable haptic substrate (including, but not limited to, e.g.,fibers, nanotubes, electroactive polymers, piezoelectric elements, orshape memory alloys). In some embodiments, the haptic output device 118is deformed, for example, with a deforming mechanism (e.g., a motorcoupled to wires), air or fluid pockets, local deformation of materials,resonant mechanical elements, piezoelectric materials,micro-electromechanical systems (“MEMS”) elements or pumps, thermalfluid pockets, variable porosity membranes, or laminar flow modulation.

Turning to memory 104, modules 124, 126, 128, and 129 are depicted toshow how a device can be configured in some embodiments to providehaptic feedback using a field of view. In some embodiments, modules 124,126, 128, and 129 may comprise processor executable instructions thatcan configure the processor 102 to perform one or more operations.

For example, a detection module 124 can configure the processor 102 tomonitor the touch sensitive surface 116 via the touch sensor 108 todetermine a position of a touch. As an example, the detection module 124may sample the touch sensor 108 in order to track the presence orabsence of a touch and, if a touch is present, to track one or more ofthe location, path, velocity, acceleration, pressure and/or othercharacteristics of the touch over time.

In some embodiments, a content provision module 129 configures theprocessor 102 to provide content (e.g., texts, images, sounds, videos,characters, virtual objects, virtual animations, etc.) to a user (e.g.,to a user of the computing device 101 or another user). If the contentincludes computer-generated images, the content provision module 129 isconfigured to generate the images for display on a display device (e.g.,the display 134 of the computing device 101, the display device 136, oranother display communicatively coupled to the processor 102). If thecontent includes video and/or still images, the content provision module129 is configured to access the video and/or still images and generateviews of the video and/or still images for display on the displaydevice. If the content includes audio content, the content provisionmodule 129 is configured to generate electronic signals that will drivea speaker, which may be part of the display 134 or display device 136,to output corresponding sounds. In some embodiments, the content, or theinformation from which the content is derived, may be obtained by thecontent provision module 129 from the storage 114, which may be part ofthe computing device 101, as illustrated in FIG. 1, or may be separatefrom the computing device 101 and communicatively coupled to thecomputing device 101. In some embodiments, the content provision module129 can cause the processor 102 to transmit content to another device.As an example, the content provision module 129 can generate or accesscontent and cause the processor to transmit the content to the displaydevice 136.

As an example, the content provision module 129 can cause the processor102 to generate a virtual environment for display on display 134 or thedisplay device 136. The virtual environment can include an environmentthat is partially or entirely virtual. For example, the virtualenvironment can include an augmented reality environment, a virtualreality environment, a video game environment, etc. As an illustrativeexample, the processor 102 can generate a virtual reality environmentassociated with a video game on the display 134 or the display device136. The virtual reality environment can include virtual objects (e.g.,characters, vehicles, buttons, sliders, knobs, icons, or other userinterface components) with which a user of the computing device 101 mayinteract. For example, the user may interact with the virtual realityenvironment using a game controller, keyboard, mouse, joystick, etc.,which can be communicatively coupled to the computing device 101 via I/Ocomponents 112. The processor 102 may receive signals via the I/Ocomponents 112 and determine an interaction with and/or manipulation ofa virtual object within the virtual reality environment based on thesignals from the I/O components 112. The processor 102 may cause theinteraction and/or manipulation to occur within the virtual environment.Thus, the user may be able to interact with or manipulate virtualobjects in the virtual reality environment.

In some embodiments, the haptic effect determination module 126represents a program component that analyzes data to determine a hapticeffect to generate. The haptic effect determination module 126 maycomprise code that selects one or more haptic effects to output usingone or more algorithms or lookup tables. In some embodiments, the hapticeffect determination module 126 comprises one or more algorithms orlookup tables usable by the processor 102 to determine a haptic effect.

Particularly, in some embodiments, the haptic effect determinationmodule 126 may determine a haptic effect based at least in part onsensor signals received from the sensor 132. For example, the sensor 132may detect an eye gaze, line-of-sight, or a direction of a field of viewof a user of the computing device 101 and transmit a sensor signal tothe processor 102. The processor 102 may receive the sensor signal anddetermine the direction of the eye gaze or field of view of the user.The haptic effect determination module 126 may determine a haptic effectbased at least in part on the determined direction of the eye gaze orfield of view of the user.

For example, in one embodiment, the haptic effect determination module126 may cause the processor 102 to access one or more lookup tables ordatabases that include data corresponding to various haptic effectsassociated with various content that can be provided by the computingdevice 101 (e.g., using the content provision module 129). The hapticeffect determination module 126 may also cause the processor 102 toaccess one or more lookup tables or databases that include datacorresponding to a position or location of various content displayed bythe computing device 101 (e.g., the position or location of the variouscontent that can be displayed via display 134 or display device 136). Inthis embodiment, the processor 102 can access the one or more lookuptables or databases and select one or more haptic effects associatedwith the content that the user is looking at based on the determineddirection of the eye gaze or field of view of the user. As an example,the processor 102 can determine that the user is looking at, or in thedirection of, a particular virtual object or character provided by thecomputing device 101 via display 134. Based on this determination, theprocessor 102 can select can select a haptic effect associated with theparticular virtual object or character. In some examples, the hapticeffect may allow the user to perceive or experience haptic effectsrelevant to the content that the user is looking at. For instance, ifthe user is looking at a car that is rewing its engine, the hapticeffect can include a vibration or a series of vibrations that can allowthe user to perceive the revving of the engine.

The processor 102 may also determine that the user is looking at, or inthe direction of, content associated with one or more haptic effects anddetermine a characteristic (e.g., a magnitude, duration, location, type,frequency, etc.) of the haptic effect based on a virtual or perceiveddistance between the user and the content. For example, in oneembodiment, the haptic effect determination module 126 may cause theprocessor 102 to access one or more lookup tables or databases thatinclude data corresponding to a perceivable or virtual distance betweena user and content (e.g., whether the user will perceive the content asbeing near the user or far from the user) and/or data corresponding tovarious parameters of the content (e.g., a size, a location on thedisplay device, an angle of display of the content, etc.) that can beused to adjust or modify the perceivable or virtual distance between theuser and the content. As an example, a virtual character may bedisplayed via display 134 as a small virtual character to cause the userto perceive the virtual character as being far from the user. In thisillustrative embodiment, the processor 102 can access the one or morelookup tables or databases and determine a virtual or perceivabledistance between the user and the content and determine a characteristicof the haptic effect based on the perceived virtual or perceiveddistance. As an example, the processor 102 can determine that the useris looking at, or in the direction of, a virtual object that is beingdisplayed via display 134 as a small virtual object such that the userperceives the virtual object as being far from the user. The processor102 can determine a weak haptic effect associated with the virtualobject based on the user perceiving the virtual object as being far.Thus, for instance, if the user is looking at a small character that isjumping up and down in the background of a scene (e.g., the userperceives the character is being far from the user), the haptic effectcan be a weak vibration.

In another embodiment, the processor 102 can determine that the user islooking at, or in the direction of, multiple content associated with oneor more haptic effects. In this embodiment, the haptic effectdetermination module 126 can cause the processor 102 to determine ahaptic effect based on each content that the user is looking at. As anexample, the processor 102 can determine that the user is looking at, orin the direction of, a first virtual character provided by the computingdevice 101 via display 134. Based on this determination, the processor102 can select a first haptic effect (e.g., a vibration) associated withthe first virtual character. The processor 102 can also determine thatthe user is looking at, or in the direction of, a second virtualcharacter provided by the computing device 101 via display 134 (e.g.,the user may be looking at the first and second virtual characterssimultaneously if, for instance, the first virtual character ispositioned near the second virtual character). Based on thisdetermination, the processor can select a second haptic effectassociated with the second virtual character. In some embodiments, thefirst haptic effect associated with the first virtual character can bedifferent from the second haptic effect associated with the secondvirtual character. In some embodiments, the processor 102 can select anddetermine a third haptic effect that includes a combination of a portionof the first haptic effect associated with the first virtual characterand a portion of the second haptic effect associated with the secondvirtual character based on the user looking at, or in the direction ofthe first and second virtual characters.

In some embodiments, the processor 102 can also select or determine acharacteristic of the first, second, or third haptic effect in responseto determining that the user is looking at, or in the direction, ofvarious content associated with one or more haptic effects. For example,the processor 102 can determine a magnitude of the first, second, orthird haptic effect based on a proportion of the user's field of viewdirected toward the first virtual character or the second virtualcharacter. As an example, the processor 102 can determine that half ofthe user's field of view is directed toward the first virtual characterand that half of the user's field of view is directed toward the secondvirtual character. Based on this determination, the processor 102 canadjust a magnitude of the first haptic effect associated with the firstvirtual character to half the magnitude and adjust a magnitude of thesecond haptic effect associated with the second virtual character tohalf the magnitude. Thus, in some examples, the characteristic of thehaptic effect perceived by the user of the computing device 101 may beproportional to, or vary depending on, the user's eye gaze,line-of-sight, or a portion of the user's field of view directed towardcontent displayed by display 134 or display device 136.

In another embodiment, the processor 102 can determine that the user islooking at, or in the direction of, a portion of the display device 136or display 134 based on sensor signals. For example, the sensor 132 maydetect that a direction of the user's eye gaze or field of view istoward a first portion of the display device 136 or display 134 andtransmit a sensor signal indicating the direction of the user's eye gazeor field of view to the processor 102. The processor 102 may receive thesensor signal and determine the direction of the eye gaze or field ofview of the user. The haptic effect determination module 126 may causethe processor 102 to access one or more lookup tables or databases thatinclude data corresponding to various haptic effects associated withvarious portions of, or locations on, the display device 136 or display134. The processor 102 can access the one or more lookup tables ordatabases and select one or more haptic effects associated with thefirst portion of the display device 136 or display 134 that the user islooking at or toward. As an example, the processor 102 can determinethat the user is looking at, or in the direction of, a left portion ofdisplay 134. Based on this determination, the processor 102 can selectcan select a haptic effect associated with the left portion of display134. In some examples, the haptic effect may allow the user of thecomputing device 101 to perceive or experience haptic effects relevantto content being displayed on a portion of the display 134 or displaydevice 136 that the user is looking at. For instance, if the user islooking at a portion of the display 134 where a collision or series ofcollisions is being displayed, the haptic effect can include a vibrationor series of vibrations corresponding to each collision, which can allowthe user to experience haptic effects associated with the one or morecollisions.

In some embodiments, the processor 102 can determine that the user islooking at, or in the direction of, multiple portions of the display 134or display device 136. In this embodiment, the haptic effectdetermination module 126 can cause the processor 102 to determine ahaptic effect based on the user looking at multiple portions of thedisplay 134 or display device 136. The processor 102 can also select ordetermine a characteristic (e.g., a magnitude, duration, location, type,frequency, etc.) of the haptic effect in response to determining thatthe user is looking at, or in the direction, of the multiple portions ofthe display 134 or display device 136.

As an example, the processor 102 can determine that the user is lookingat, or in the direction of, a first portion of the display 134. Theprocessor 102 can also determine that the user is looking at, or in thedirection of, a second portion of the display 134. For example, thefirst portion can be the top portion of the display 134, the secondportion can be the left portion of the display 134, and the user can besimultaneously looking at the first and second portions (e.g., lookingat a top-left portion of the display 134). The processor 102 candetermine a haptic effect (e.g., a vibration) that can include one ormore haptic effects based on the user looking at the first and secondportions of the display 134. For example, the processor 102 can select ahaptic effect associated with the top portion, left portion, or atop-left portion of the display 134. In some examples, the processor 102can select a first haptic effect associated with the top portion and asecond haptic effect associated with the left portion. In some examples,the haptic effect associated with the top portion of the display 134 canbe different from the haptic effect associated with the left portion ofthe display 134.

In some embodiments, the processor 102 can determine a magnitude of thehaptic effect based on a proportion of the user's field of view directedtoward the top portion or the left portion of the display 134. As anexample, the processor 102 can determine that one-third of the user'sfield of view is directed toward the top portion of the display 134 andthat two-thirds of the user's field of view is directed toward the leftportion of the display 134. Based on this determination, the processor102 can adjust a magnitude of the haptic effect associated with the topportion to one-third of the magnitude and adjust a magnitude of thehaptic effect associated with the left portion to two-thirds of themagnitude. Thus, in some examples, the haptic effect or a characteristicof the haptic effect perceived by the user of the computing device 101may be proportional to, or vary depending on, a proportion of the user'seye gaze, line-of-sight, or field of view directed toward a portion ofthe display 134 or display device 136.

In this manner, the system 100 for haptic feedback based on a field ofview can provide a user with one or more haptic effects relevant toparticular content that the user is looking at or relevant to contentdisplayed on a particular portion of a display device that the user islooking at, which can provide the user with a more immersive experienceas the user views the content on the display device. Thus, for instance,the user can perceive haptic effects that are relevant to the contentthat the user is looking at so that the user is not overwhelmed withhaptic effects associated with content that the user is not looking ator toward.

In another embodiment, the processor 102 can receive sensor signals fromthe sensor 132 and determine a distance between the sensor 132, thecomputing device 101, or a user of the computing device 101, and thedisplay 134 or display device 136 (e.g., based on the strength of theBluetooth signal between in the sensor 132 and the display 134 ordisplay device 136). In this embodiment, the haptic effect determinationmodule 126 can cause the processor 102 to determine a haptic effectbased at least in part on the determined distance. The haptic effectdetermination module 126 may also cause the processor 102 to select ordetermine a characteristic (e.g., a magnitude, duration, location, type,frequency, etc.) of the haptic effect based at least in part on thedetermined distance.

As an example, the processor 102 can determine that the user is neardisplay device 136 or content displayed on display device 136. Theprocessor 102 can determine a haptic effect based on the user being nearthe display device 136 or content displayed on display device 136. Forexample, the haptic effect can be a strong or long haptic effect if theuser is near the display device 136. As another example, the processor102 can determine that the user is far from the display device 136 orcontent displayed on the display device 136 and determine a weak orshort haptic effect based on this determination.

In another embodiment, the haptic effect determination module 126 maycomprise code that determines a haptic effect based on content providedby the content provision module 129. For example, the content provisionmodule 129 may provide visual content to be output on the display device136 or display 134. In one embodiment, the haptic effect determinationmodule 126 may determine a haptic effect associated with the visualcontent. For example, in one such embodiment, the haptic effectdetermination module 126 may determine a haptic effect for providing ahaptic track associated with a video being provided by the displaydevice 136. A haptic track can include a haptic effect (e.g., avibration) or a series of haptic effects that correspond to the eventsoccurring in the video being provided. For instance, if the videoincludes a series of explosions, the haptic track can be a series ofvibrations that correspond to each explosion. Thus, as the user watchesthe video, the user may perceive the haptic effects associated with thevideo.

In some embodiments, the haptic effect determination module 126 maycomprise code that determines, based on a location of a touch on thetouch sensitive surface 116, a haptic effect to output and code thatselects one or more haptic effects to provide in order to simulate theeffect. For example, different haptic effects may be selected based onthe location of a touch in order to simulate the presence of a virtualobject (e.g., a virtual piece of furniture, automobile, animal, cartooncharacter, button, lever, logo, or person) on the display 134. Further,in some embodiments, haptic effect determination module 126 may comprisecode that determines, based on the size, color, location, movement,and/or other characteristics of a virtual object, a haptic effect tooutput and code that selects one or more haptic effects to provide inorder to simulate the effect. For example, haptic effects may beselected based on the color of a virtual object (e.g., a strongvibration if the virtual object is red, and a weaker vibration if thevirtual object is green).

In some embodiments, the haptic effect determination module 126comprises code that determines a haptic effect based on an event. Anevent, as used herein, is any interaction, action, collision, or otherevent, which occurs during operation of the computing device 101, whichcan potentially comprise an associated haptic effect. In someembodiments, an event may comprise user input (e.g., a button press,manipulating a joystick, interacting with a touch sensitive surface 116,tilting or orienting the device), a system status (e.g., low battery,low memory, or a system notification, such as a notification generatedbased on the system receiving a message, an incoming phone call, anotification, or an update), sending data, receiving data, or a programevent (e.g., if the program is a game, a program event may compriseexplosions, gunshots, collisions, interactions between game characters,advancing to a new level, or driving over bumpy terrain).

In some embodiments, the haptic effect determination module 126 cancause the processor 102 to select one or more haptic effects to provide,and/or one or more haptic output devices 118 to actuate, in order togenerate or output the haptic effect.

In some embodiments, the haptic effect generation module 128 representsprogramming that causes the processor 102 to generate and transmithaptic signals to a haptic output device 118, 159 to generate theselected haptic effect. In some examples, the haptic effect generationmodule 128 causes the haptic output device 118 to generate a hapticeffect determined by the haptic effect determination module 126. Forexample, the haptic effect generation module 128 may access storedwaveforms or commands to send to the haptic output device 118 to createthe selected haptic effect. For example, the haptic effect generationmodule 128 may cause the processor 102 to access a lookup table thatincludes data indicating one or more haptic signals associated with oneor more haptic effects and determine a waveform to transmit to hapticoutput device 118 to generate a particular haptic effect. In someembodiments, the haptic effect generation module 128 may comprisealgorithms to determine the haptic signal. The haptic effect generationmodule 128 may comprise algorithms to determine target coordinates forthe haptic effect (e.g., coordinates for a location on the computingdevice 101, such as on the touch sensitive surface 116, at which tooutput the haptic effect). For example, the haptic effect generationmodule 128 may cause the processor 102 to use a sensor signal indicatinga location of a touch of an object on the touch sensitive surface 116 todetermine target coordinates for the haptic effect, which may correspondto the location of the touch. In some embodiments, the haptic outputdevice 118 may include one or more haptic output devices. In suchembodiments, the haptic effect generation module 128 may cause theprocessor 102 to transmit haptic signals to the one or more hapticoutput devices 118 to generate the selected haptic effect.

In some embodiments, the display device 136 comprises, for example, amobile device (e.g., a smartphone), tablet, e-reader, smartwatch, ahead-mounted display, a wearable device, a video screen, a headset, avirtual reality display device, or any other device that includes adisplay for providing content.

The display device 136 may include a processor 138, a memory 140, a bus142, I/O components 144, storage 146, network interface device 148,display 150, touch sensitive surface 152, touch sensors 154, camera 156,sensor 158, and haptic output device 159, each of which may beconfigured in substantially the same manner as the processor 102, memory104, bus 106, I/O components 112, storage 114, network interface device110, display 134, touch sensitive surface 116, touch sensors 108, camera130, sensors 132, and haptic output device 118 of the computing device101, although they need not be. In some embodiments, the computingdevice or the display device 136 may include all or some of thecomponents depicted in FIG. 1. As an example, the computing device 101can be a smartphone and the display device 136 can be a tablet that auser of the computing device 101 is looking at or toward. In such anexample, the computing device 101 or the display device 136 can includeall or some of the components depicted in FIG. 1.

The computing device 101 may be communicatively coupled to the displaydevice 136. For example, the computing device 101 and the display device136 can communicate (e.g., transmit or receive data or signals 172, 174)using network interface devices 110 and 148. As an example, thecomputing device 101 and the display device 136 can each be connected toa common wireless network and can communicate via the wireless network.As another example, the computing device 101 and the display device 136can be communicatively coupled via a Bluetooth connection.

In some embodiments, the display device 136 may be a display device thatcan be used to provide content to a user of the computing device 101 ordisplay device 136. For example, the display device 136 can be used toprovide texts, images, sounds, videos, characters, virtual objects,virtual animations, etc. As an example, the display device 136 can beused to provide a virtual environment (e.g., an augmented realityenvironment, a virtual reality environment, a video game environment,etc.) to the user. As an illustrative example, the display device 136may provide a virtual reality environment that includes virtual objects(e.g., characters, vehicles, buttons, sliders, knobs, icons, or otheruser interface components) with which the user may interact. Forexample, the user may interact with the virtual reality environmentusing the computing device 101 or one or more game controllerscommunicatively coupled to the computing device 101 or display device136 via I/O components 112, 144. The processors 102, 138 may receivesignals from the game controllers via the I/O components 112, 144 anddetermine an interaction with and/or manipulation of a virtual objectwithin the virtual reality environment based on the signals from the I/Ocomponents 112, 144. The processor 102, 138 may cause the interactionand/or manipulation to occur within the virtual environment. As anillustrative example, the computing device 101 can be a smartphone andthe display device 136 can be a virtual reality display device that isdisplaying one or more virtual objects. A user of the smartphone may belooking at or toward the virtual objects displayed by the virtualreality display device and the user may interact with or control thevirtual objects that the user is looking at by providing one or moreuser inputs to the smartphone, which can be communicated to the virtualreality display device by the smartphone. Thus, the user may be able tointeract with or manipulate virtual objects in a virtual realityenvironment.

In some embodiments, the sensor 158 of the display device 136 may detectan eye gaze, line-of-sight, or field of view of a user of the computingdevice 101 or display device 136. For example, the sensor 158 may beconfigured in substantially the same manner as sensor 132 for detectingthe eye gaze, line-of-sight, or field of view of a user of the computingdevice 101 or display device 136.

In some embodiments, the memory 140 includes modules 160, 162, 164, 166.In some embodiments, the memory 140 includes all or some of the modules160, 162, 164, 166. The modules 160, 162, 164, 166 can each beconfigured in substantially the same manner as respective modules 124,126, 128, and 129 of the computing device 101.

In some embodiments, the content provision module 166 can be configuredin substantially the same manner as the content provision module 129 andcan configure the processor 138 to provide content (e.g., texts, images,sounds, videos, animated graphics, virtual objects, virtual animations,etc.) to a user (e.g., to a user of the computing device 101 or anotheruser). In some embodiments, the content, or the information from whichthe content is derived, may be obtained from another device (e.g., thecomputing device 101). For example, the content provision module 166 canbe configured to cause the processor 138 to receive data that includescontent from the computing device 101. The content may displayed by thedisplay device 136 (e.g., via the display 150).

In some embodiments, the haptic effect determination module 162 maycause the processor 138 to determine a haptic effect based at least inpart on sensor signals received from the sensor 158 in substantially thesame manner as described above with respect to haptic effectdetermination module 126 and processor 102. For example, the hapticeffect determination module 162 may cause the processor 138 to determineone or more haptic effects based on the direction of the eye gaze orfield of view of the user of the computing device 101. As an example,the haptic effect determination module 162 may cause the processor 138to determine one or more haptic effects in response to determining thatthe user is looking at, or in the direction of, content associated witha haptic effect and displayed on display device 136. As another examplethe haptic effect determination module 162 may cause the processor 138to determine one or more haptic effects in response to determining thatthe user is looking at, or in the direction of, a portion of the displaydevice 136. As another example, the haptic effect determination module162 can cause the processor 138 to determine a characteristic of thehaptic effect based on the direction of the eye gaze or field of view ofthe user of the computing device 101 or display device 136. In anotherembodiment, the haptic effect determination module 162 can cause theprocessor 138 to determine a characteristic of the haptic effect basedon a virtual or perceived distance between the user and the content. Inanother embodiment, the haptic effect determination module 162 can causethe processor 138 to determine a characteristic of the haptic effectbased on a distance between a user of the computing device 101 ordisplay device 136 and the display device 136 or sensor 158. Further, insome embodiments, the haptic effect determination module 162 can causethe processor 102 to select one or more haptic effects to provide,and/or one or more haptic output devices 118 to actuate, in order togenerate or output the haptic effect.

In some embodiments, the haptic effect generation module 164 representsprogramming that causes the processor 138 to generate and transmithaptic signals to haptic output device 118 or 159 to generate theselected haptic effect. In some examples, the haptic effect generationmodule 164 causes the haptic output device 118, 159 to generate a hapticeffect determined by the haptic effect determination module 162.

As an illustrative example, the computing device 101 can be a smartwatchand the display device 136 can be a tablet that a user wearing thesmartwatch is looking at. In one such example, the tablet can displaycontent to the user via a display of the tablet (e.g., using the contentprovision module 166 to provide content via display 150) and the sensor158 can determine a direction of the eye gaze of the user. The processorof the tablet (e.g., processor 138) can determine a haptic effect inresponse to determining that the user is looking at content displayed bythe tablet and associated with a haptic effect. The processor of thetablet can generate and transmit a haptic signal to a haptic device ofthe smartwatch (e.g., haptic device 118) or a haptic device of thetablet (e.g., haptic output device 159) to generate a haptic effect tobe perceived by the user. For example, the haptic effect may be outputby the smartwatch or the tablet and the user may perceive the hapticeffect as the user is wearing the smartwatch or if the user is incontact with the tablet (e.g., touching or holding the tablet).

FIG. 2 shows an embodiment of a system 200 for haptic feedback using afield of view. The system 200 includes computing devices 101 a-b (e.g.,the computing device 101 of FIG. 1) and a display device 136.

In the example shown in FIG. 2, the display device 136 can be anydisplay device (e.g., a video screen) configured to display content tousers 202 a-b of computing devices 101 a-b. For example, the processorof the display device 136 may generate content to be displayed bydisplay device 136. In another example, the processor of the computingdevice 101 (e.g., the processor 102 of FIG. 1) may generate content andtransmit data that includes content to the display device 136 and thedisplay device 136 can be configured to display the content.

In some embodiments, the display device 136 or the computing devices 101a-b may determine one or more haptic effects associated with the contentdisplayed on the display device 136. For example, the content maycomprise a video, or a video game, and the computing devices 101 a-b mayoutput one or more haptic effects (e.g., via the haptic output device118 of FIG. 1) that correspond to the video game or video contentdisplayed on the display device 136. As an example, the video contentmay include one or more actions or interactions (e.g., interactionbetween characters or objects in the video displayed by display device136). The computing devices 101 a-b can output one or more vibrationsassociated with the actions or interaction so that the users 202 a-b mayperceive the actions and/or interactions as they are displayed on thedisplay device 136.

In some embodiments, the display device 136 comprises a sensor 203,which may be configured in substantially the same manner as the camera156 or the sensor 158 of FIG. 1. In some embodiments, the sensor 203 maybe internal to the display device 136 or external to the display device136 and communicatively coupled to the display device 136 (e.g., via awired or wireless communication link). The sensor 203 may be configuredto detect a line-of-sight or field of view 210 of the user 202 a or adirection of an eye gaze 212 a-b of the user 202 a. The sensor 158 mayalso detect a line-of-sight or field of view 214 of the user 202 b.

In some embodiments, sensor 203 may transmit data indicating a detectedline-of-sight or field of view 210 of the user 202 a, the direction ofan eye gaze 212 a-b of the user 202 a, or the line-of-sight or field ofview 214 of the user 202 b. For example, the sensor 203 may detect afield of view 210 of the user 202 a or the direction of the eye gaze 212a-b of the user 202 a. The sensor 203 may transmit a sensor signalindicating the field of view 210 of the user 202 a or the direction ofthe eye gaze 212 a-b to a processor of the display device 136 (e.g., theprocessor 138 of FIG. 1), which may determine the field of view 210 ofthe user 202 a or the direction of the eye gaze 212 a-b based on thedata.

In some embodiments, the display device 136 may determine one or morehaptic effects based at least in part on the sensor signal from thesensor 203. The haptic effects may each comprise a vibration or anyother haptic effect. For example, in some embodiments, the displaydevice 136 may determine a haptic effect based on the field of view 210of the user 202 a or the direction of the eye gaze 212 a-b of the user202 a. In one such embodiment, the display device 136 determines thatthe user 202 a is looking at, or in the direction of, virtual objects204, 206 displayed via display device 136, based on the data from thesensor 203 and may determine a first haptic effect (e.g., a vibration)associated with virtual object 204 and a second haptic effect (e.g.,another vibration) associated with virtual object 206. In someembodiments, the first haptic effect can be different from the secondhaptic effect. The display device 136 may transmit a first haptic signalcorresponding to the first haptic effect and/or a second haptic signalcorresponding to the second haptic effect to a haptic output deviceassociated with the user 202 a. For example, the display device 136 maytransmit the first or second haptic signal to the computing device 101 aassociated with the user, which includes a haptic output device (e.g.,the haptic output device 118 of FIG. 1). The haptic output device canoutput the first and second haptic effects to the user 202 a (e.g., to awrist, hand, arm, leg, or hand of the user 202 a). The first or secondhaptic effect may allow the user to perceive or experience hapticeffects relevant to the virtual objects 204, 206. For example, ifvirtual object 204 is a virtual character that is jumping up and down,the first haptic effect associated with the virtual object 204 caninclude one or more vibrations that allow the user 202 a to perceive theeffects of the virtual object 204 jumping up and down.

The display device 136 may also determine a characteristic (e.g., amagnitude, duration, location, type, frequency, etc.) of the one or morehaptic effects based at least in part on a size, color, location,movement, and/or other characteristic of the virtual objects 204, 206,208. For example, in the example depicted in FIG. 2, the virtual objects204, 206 are within the field of view 210 of the user 202 a. The virtualobject 206 may be displayed on display device 136 as a three-dimensional(“3D”) virtual object. In this example, the display device 136 maydetermine a strong or long first haptic effect (e.g., a strong or longvibration) associated with the virtual object 206 based on the virtualobject 206 being displayed as a 3D virtual object. The display device136 can transmit a first haptic signal associated with the first hapticeffect to the computing device 101 a and the haptic output device of thecomputing device 101 a can output the haptic effect to the user 202 a.Thus, in some examples, the characteristic of the haptic effectperceived by the user 202 a may be proportional to, or vary dependingon, a characteristic of the virtual objects 204, 206 within the field ofview 210 of the user 202 a.

In some embodiments, the display device 136 may determine acharacteristic of the one or more haptic effects based on the eye gaze212 a-b of the user 202 a-b. For example, the display device 136 candetermine a characteristic of the haptic effects based on a change inthe eye gaze 212 a-b of the users 202 a-b.

For example, in the example depicted in FIG. 2, the virtual objects 204,206 are within the field of view 210 of the user 202 a, and the eye gazeof the user 202 a may pan or move between the virtual objects 204, 206.In this example, the computing device 101 a may include a first hapticoutput device and a second haptic output device. In some examples, thefirst and second haptic output devices can be combined to form a singlehaptic output device. The first haptic output device can be configuredto output a haptic effect to a first part of the user (e.g., a rightwrist, right hand, right arm, or a right side of the head of the user202 a). And the second haptic output device can be configured to outputa haptic effect to a second part of the user (e.g., a left wrist, lefthand, left arm, or a left side of the head of the user 202 a). The eyegaze 212 a of the user 202 a may initially be directed toward thevirtual object 206, which may be displayed on the display device 136toward the right of the user 202 a. The display device 136 may transmita first haptic signal corresponding to a first haptic effect to thefirst haptic output device of the computing device 101 a in response tothe eye gaze 212 a of the user 202 a being directed toward the virtualobject 206. The first haptic output device can output the first hapticeffect to a right side of the user 202 a (e.g., a right arm of the user202 a). In some examples, the first haptic effect can be a strong hapticeffect (e.g., a strong vibration) in response to the eye gaze 212 a ofthe user 202 a being directed toward the virtual object 206.

Subsequently, the eye gaze of the user 202 a may shift such that eyegaze 212 b of the user 202 a may be directed toward the virtual object204, which may be displayed on the display device 136 toward the left ofthe user 202 a. As the eye gaze of the user 202 a shifts from thevirtual object 206 to the virtual object 204 (e.g., from eye gaze 212 ato 212 b), the display device 136 may transmit a second haptic signalcorresponding to a second haptic effect to the second haptic outputdevice of the computing device 101. The second haptic output device canoutput the second haptic effect to a left side of the user 202 a (e.g.,the left arm of the user 202 a).

In this example, the display device 136 can adjust a characteristic ofthe first haptic effect as the eye gaze of the user 202 a shifts fromeye gaze 212 a to eye gaze 212 b. For example, the display device 136may reduce a magnitude of the first haptic effect and/or increase amagnitude of the second haptic effect as the eye gaze of the user shiftsfrom the virtual object 206 to the virtual object 204 (e.g., from eyegaze 212 a to 212 b). In some examples, the first haptic effect cancomprise a single haptic track. For example, the haptic track can be aseries of haptic effects (e.g., a series of vibrations) that includesthe first haptic effect and the second haptic effect. In this example,the display device 136 can adjust a characteristic (e.g., a magnitude)of the haptic track as the eye gaze of the user 202 a shifts from eyegaze 212 a to eye gaze 212 b as described above.

Thus, in some examples, a characteristic of a haptic effect perceived bya user 202 a-b may vary as the eye gaze of the user 202 a-b changes. Inanother example, the user 202 a-b may perceive various haptic effectscorresponding to a horizontal and/or vertical position of the contentthat the user 202 a-b is looking at on the display device 136. Forexample, as described above, the user 202 a may perceive a haptic effecton the left side of the user 202 a if the user 202 a is looking atcontent displayed on the display device 136 to the left of the user 202a.

In some embodiments, the display device 136 may determine acharacteristic of one or more haptic effects based on a virtual orperceived distance between the user and the content that the user islooking at, in the direction of, or toward. For example, in the exampledepicted in FIG. 2, the virtual objects 204, 206 are within the field ofview 210 of the user 202 a. The virtual object 206 may be displayed ondisplay device 136 as a smaller virtual object than virtual object 204,which may cause the user 202 a to perceive the virtual object 206 asbeing far and to perceive the virtual object 204 as being close. In thisexample, the display device 136 may determine a strong or long firsthaptic effect (e.g., a strong or long vibration) associated with thevirtual object 204. As another example, the display device 136 maydetermine a weak or short second haptic effect (e.g., a weak or shortvibration) associated with the virtual object 206. The display device136 can transmit a first haptic signal associated with the first hapticeffect or a second haptic signal associated with the second hapticeffect to the computing device 101 a and the haptic output device of thecomputing device 101 a can output the haptic effects to the user 202 a.Thus, in some examples, the characteristic of the haptic effectperceived by the user 202 a may be proportional to, or vary dependingon, a virtual or perceived distance between the user 202 a and virtualobjects 204, 206 within the field of view 210 of the user 202 a.

In another embodiment, the sensor signal from the sensor 203 canindicate that the user 202 a is looking at, or in the direction of,multiple virtual objects 204, 206. For example, in the example depictedin FIG. 2, data from sensor 203 can indicate that a direction of an eyegaze 212 a of the user 202 a is toward, or in the direction of, virtualobject 206 and that a direction of an eye gaze 212 b of the user 202 ais toward, or in the direction of, virtual object 204. For instance, thevirtual object 204 can be near the virtual object 206 and the user 202 amay be looking at virtual objects 204, 206 simultaneously. In one suchembodiment, the display device 136 can determine a third haptic effect.In some embodiments, the third haptic effect may be different from thefirst and second haptic effects. In another embodiment, the third hapticeffect can include a combination of the first and second haptic effects.For example, the third haptic effect may include a portion of the firsthaptic effect associated with the virtual object 204 and a portion ofthe second haptic effect associated with the virtual object 206.

In some embodiments, the display device 136 can determine acharacteristic of the first haptic effect associated with the virtualobject 204 and a characteristic of the second haptic effect associatedwith the virtual object 206 based on the direction of the eye gaze 212a-b of the user 202 a. For example, the display device 136 can determinethat a portion of the user's field of view 210 corresponding to thedirection of the eye gaze 212 a of the user 202 a is greater than aportion of the user's field of view 210 corresponding to the directionof the eye gaze 212 b of the user 202 a. For example, the display device136 may determine that three-fourths of the user's field of view 210 isdirected toward the virtual object 206 based on the direction of the eyegaze 212 a of the user 202 a. The display device 136 may also determinethat one-fourth of the user's field of view 210 is directed toward thevirtual object 204 based on the direction of the eye gaze 212 b of theuser 202 a. Based on this determination, the display device 136 canadjust a magnitude of the first haptic effect associated with thevirtual object 204 to one-fourth of the magnitude. The display device136 can also adjust a magnitude of the second haptic effect associatedwith the virtual object 206 to three-fourths of the magnitude. In thismanner, the user 202 a may perceive the second haptic effect more thanthe first haptic effect if the user 202 a is more focused on the virtualobject 206 than the virtual object 204. Thus, in some examples, thecharacteristic of the haptic effect perceived by the user 202 a may beproportional to, or vary depending on, the eye gaze 212 a-b of the user202 a or a portion of the user's field of view 210 directed towardvirtual objects 204, 206.

In some embodiments, the sensor signal from the sensor 203 can indicatea distance between the user 202 a and the display device 136 or virtualobjects 204, 206 within the field of view 210 of the user 202 a. In onesuch embodiment, the display device 136 may determine a haptic effect ora characteristic of the haptic effect based on the distance between theuser 202 a and the display device 136 or virtual objects 204, 206. As anexample, the display device 136 may determine a strong or long firsthaptic effect (e.g., a strong or long vibration) associated with thevirtual object 204 if the user 202 a is near the virtual object 204. Asanother example, the display device 136 may determine a weak or shortsecond haptic effect associated with the virtual object 206 if the user202 a is far from the virtual object 206. The display device 136 cantransmit a haptic signal associated with the haptic effect to the hapticoutput device associated with the user 202 a, which can output thehaptic effect to the user 202 a.

In the example depicted in FIG. 2, the field of view 214 of the user 202b is directed toward virtual object 208. The sensor 203 may transmitsensor signals indicating the direction of the field of view 214 and thedisplay device 136 can determine a haptic effect associated with thevirtual object 208. The display device 136 can transmit a haptic signalassociated with the haptic effect to a haptic output device associatedwith the user 202 b, which can output the haptic effect to the user 202b.

In some embodiments, the users 202 a-b may not perceive a haptic effectassociated with a virtual object 204, 206, 208 that is outside of thefield of view 210, 214 of the users 202 a-b. For example, data fromsensor 203 may indicate that virtual objects 204, 206 are outside thefield of view 214 of the user 202 b (e.g., the user 202 b is not lookingat, toward, or in the direction of virtual objects 204, 206). In onesuch embodiment, a haptic effect associated with virtual objects 204,206 may not be output to the user 202 b. In this manner, sensor signalsfrom the sensor 203 can be used to provide haptic effects to users 202a-b based on the field of view 210, 214 of the users 202 a-b, which canprovide the users 202 a-b with a more immersive experience relative tocontent that the users 202 a-b are looking at by allowing the users 202a-b to perceive or experience haptic effects relevant to the particularcontent that the users 202 a-b are looking at or toward.

In some embodiments, the display device 136 may determine one or morehaptic effects based on sensor signals from the sensor 203 as describedabove. In another embodiment, the sensor 203 can transmit sensor signalsto computing devices 101 a-b, each of which can determine one or morehaptic effects based on sensor signals in substantially the same manneras described above with reference to the display device 136. Forexample, the sensor 203 can transmit a sensor signal indicating thefield of view 210, 214 of the users 202 a-b or the direction of the eyegaze 212 a-b of the user 202 a to a processor of each computing device101 a-b (e.g., the processor 102 of FIG. 1) and the computing devices101 a-b can determine one or more haptic effects based on the sensorsignal. In such embodiments, the computing devices 101 a-b may determinea haptic effect based on the sensor signal from the sensor 203 andtransmit a haptic signal associated with the haptic effect to a hapticoutput device associated with the users 202 a-b (e.g., haptic outputdevice 118 of FIG. 1), which can output the haptic output effect to theusers 202 a-b.

While in this example, the users 202 a-b may not perceive haptic effectsassociated with a virtual object 204, 206, 208 that is outside of thefield of view 210, 214 of the users 202 a-b, the present disclosure isnot limited to such configurations. Rather, in other examples, such as,for example, embodiments described in further detail below, the displaydevice 136 or the computing device 101 a-b may determine and output oneor more haptic effects based on content outside of the field of view210, 214 of the users 202 a-b.

FIG. 3 shows another embodiment of a system 300 for haptic feedbackusing a field of view according to another embodiment. In the exampledepicted in FIG. 3, the system 300 comprises a computing device 302,such as a smartphone, a tablet, or a game controller, and a displaydevice 304. The display device 304 depicted in FIG. 3 can be a wearabledevice (e.g., a virtual reality headset).

In this embodiment, the display device 304 generates and provides avirtual environment (e.g., a virtual reality environment associated witha video game) to a user 306 of the display device 304. The virtualenvironment includes one or more virtual objects (e.g., characters,vehicles, buttons, sliders, knobs, icons, or other user interfacecomponents) with which the user 306 can interact using the computingdevice 302. As an example, the computing device 302 can include one ormore buttons, joysticks, etc. communicatively coupled to the computingdevice 302 via one or more I/O components (e.g., I/O components 112 ofFIG. 1). The computing device 302 can be communicatively coupled to thedisplay device 304 to allow the user 306 interact with the virtualenvironments via the I/O components included in the computing device302.

In some embodiments, the display device 304 comprises a sensor (notshown), which may detect a field of view of the user 306, or a directionof an eye gaze of the user 306 (e.g., when the user 306 is looking in ata particular portion of the display device 304 or at a particularvirtual object displayed by display device 304). For example, the sensormay be a camera or other sensor configured in substantially the samemanner as sensor 132 or 158 of FIG. 1. As another example, the sensorcan detect a motion of the head of the user 306 or the head posture ofthe user 306 for determining a field of view of the user 306, or thedirection of the eye gaze of the user 306. The sensor can transmitsensor signals to a processor of the display device 304 (not shown) or aprocessor of the computing device 302 (not shown), which may determinethat the field of view or direction of an eye gaze of the user 306corresponds with a location or position of a virtual object displayedvia display device 304. In some embodiments, the display device 304 orthe computing device 302 may determine one or more haptic effects (e.g.,a vibration, a squeeze, or a poke) associated with the virtual objectand output the haptic effect to a body part of the user 306 (e.g., theuser's head, hand, or other body part). In some embodiments, outputtinga haptic effect associated with a virtual object within the field ofview or direction of an eye gaze of the user 306 can allow the user toperceive haptic effects relevant to a virtual object that the user 306is looking at. As an example, the display device 304 is displayingmultiple virtual characters and the user 306 is looking at a particularvirtual character engaged in combat. The sensor signals can indicatethat the user is looking at the particular virtual character and thedisplay device 304 can output a haptic effect associated with thevirtual character being engaged in combat (e.g., via haptic outputdevice 159 of FIG. 1). The haptic effect can include a vibration or aseries of vibrations to allow the user 306 to perceive haptic effectsassociated with the particular virtual character that the user 306 islooking at.

In some embodiments, the display device 304 can determine that the useris looking at multiple virtual objects displayed on display device 304and the display device 304 can determine a haptic effect associated witheach virtual object or a characteristic of the haptic effect associatedwith each virtual object based on this determination. As an example, theuser 306 is looking at a first virtual character and a second virtualcharacter simultaneously. The sensor signals indicate that an eye gazeof the user 306 is directed at the first virtual character and thesecond virtual character is within the field of view of the user 306.The display device 304 can determine and output one or more hapticeffects that include a portion of a first haptic effect associated withthe first virtual character and a portion of a second haptic effectassociated with the second virtual character. In this manner, the user306 may experience haptic effects associated with both the first andsecond virtual characters that the user 306 is looking at.

The display device 304 may also determine a characteristic of the hapticeffect based on a virtual or perceived distance between the user 306 anda particular virtual object that the user 306 is looking at or toward.As an example, the display device 304 is displaying a war scene thatincludes multiple virtual characters engaged in combat. In this example,a first virtual character is engaged in combat and is displayed on thedisplay device 304 such that the user 306 perceives the first virtualcharacter as being near the user 306 (e.g., the first virtual characteris displayed as a large virtual character so that the user 306 perceivesthe first virtual character as being close to the user 306). A secondvirtual character is engaged in combat in a background of the war sceneand displayed on the display device 304 such that the user 306 perceivesthe second virtual character as being far from the user 306 (e.g., thesecond virtual character is displayed as a small virtual character sothat the user 306 perceives the second virtual character as being farfrom the user 306). The display device 304 can determine a first hapticeffect (e.g., a strong vibration) associated with the first virtualcharacter based on the user 306 perceiving the first virtual characteras being near the user 306. The display device 304 may also determine asecond haptic effect (e.g., a weak vibration) associated with the secondvirtual character based on the user 306 perceiving the second virtualcharacter as being far from the user 306.

In some embodiments, the display device 304 can determine that the user306 is looking at a portion of the display device 304 based on sensordata (e.g., data from the sensor of the display device 304). The displaydevice 304 may determine one or more haptic effects associated with theportion of the display device 304 that the user 306 is looking at. As anexample, the display device 304 is divided into a left portion, a middleportion, and a right portion (not shown). The sensor data indicates thatthe user 306 is looking at the right portion of the display device 304,which includes a scene involving a series of collisions. The displaydevice 304 can output a haptic effect associated with the right portionof the display device 304 and the series of collisions displayed on theright portion (e.g., a series of vibrations) to allow the user 306 toexperience haptic effects associated with the collisions displayed onthe right portion of the display device 304. In the embodiment depictedin FIG. 3, the display device 304 comprises a wearable device (e.g., avirtual reality headset) and may comprise one or more haptic outputdevices. The display device 304 may be configured to output the hapticeffect to a right side of the display device 304 or to a right side ofthe body of the user 306 (e.g., to the right side of the head of user306) in response to determining that the user 306 is looking at theright portion of the display device 304.

In some examples, the display device 304 can include one or more sensorsor emitters that can each be configured in substantially the same manneras sensor 203 of FIG. 2. The various sensors or emitters can bepositioned at various locations within or on the display device 304 andthe locations of the sensors may correspond to various portions of thedisplay device 304. As an example, a first sensor can be positioned neara left portion of the display device 304, a second sensor can bepositioned near a middle portion of the display device 304, and a thirdsensor can be positioned near a right portion of the display device 304.Each sensor can detect a motion of the head of the user 306, the headposture of the user 306, a field of view of the user 306, or a directionof an eye gaze of the user 306. In some such embodiments, each sensorcan transmit sensor signals to a processor of the display device 304(not shown) or a processor of the computing device 302 (not shown),which may determine that the field of view or direction of an eye gazeof the user 306 corresponds with a location or position of the sensorand a portion of the display device 304 associated with sensor (e.g., aportion of the display device 304 near the sensor). The display device304 or the computing device 302 may determine one or more haptic effects(e.g., a vibration, a squeeze, or a poke) associated with the portion ofthe display device 304 associated with the sensor and output the hapticeffect to a body part of the user 306. In this manner, the displaydevice 304 can determine and output haptic effects based on spatialdata, such as, for example, a location of a sensor relative to a portionof the display device 304, and the direction of the eye gaze or field ofview of the user 306 corresponding to the location of the sensor.

As an example, the display device 304 is divided into a left portion, amiddle portion, and a right portion. A sensor positioned near the leftportion detects the direction of the eye gaze of the user 306 andtransmits sensor data indicating that the user 306 is looking at ortoward the left portion of the display device 304, which includes ascene depicting a series of explosions. The display device 304 canoutput a haptic effect associated with the left portion of the displaydevice 304 and the series of explosions displayed on the left portion(e.g., a series of vibrations) to allow the user 306 to experiencehaptic effects associated with the explosions displayed on the leftportion of the display device 304. In this manner, the display device306 can use the sensor to detect the eye gaze or direction of field ofview of the user 306 and use spatial data about the sensor (e.g.,coordinates of a location of the sensor or a position of the sensorrelative to a portion of the display device 304) to determine one ormore haptic effects that are relevant to the portion of the displaydevice that the user 306 is looking at or toward.

In some embodiments, the display device 304 can determine that the useris simultaneously looking at more than one portion of the display device304 and the display device 304 can determine a haptic effect associatedwith each portion of the display device 304 or a characteristic of eachhaptic effect based on this determination. As an example, the user 306is looking at the left portion of the display device 304, which includesa scene involving a virtual character engaged in combat. The user 306 issimultaneously looking at the middle portion of the display device 304,which includes a scene involving a car revving its engine. The sensorsignals indicate that a field of view of the user 306 is directed towardthe left and middle portions. The display device 304 can determine andoutput one or more haptic effects based on the sensor signal. Forexample, the display device 304 can output a first haptic effectassociated with the left portion (e.g., a vibration corresponding to thevirtual character being engaged in combat). The display device 304 canoutput a second haptic effect associated with the middle portion (e.g.,a series of vibrations corresponding to the car rewing its engine). Insome embodiments, a single haptic effect may comprise the first andsecond haptic effects. In this manner, the user 306 may experiencehaptic effects associated with both the left and middle portions of thedisplay device 304 that the user 306 is looking at.

In the example depicted in FIG. 3, the display device 304 may determineand output one or more haptic effects based on sensor data as describedabove. In another embodiment, the display device 304 may determine oneor more haptic effects and transmit a haptic signal associated with thehaptic effects to the computing device 302, which can include a hapticoutput device configured to output the haptic effect to the user 306. Instill another embodiment, the sensor of the display device 304 maytransmit sensor signals to the computing device 302, which can determineone or more haptic effects based on the sensor signals and output thehaptic effects to the user 306 in substantially the same manner asdescribed above with respect to the display device 304.

While in the example described in FIG. 3, the display device 304 candetermine one or more haptic effects based on the field of view of theuser 306 or a direction of an eye gaze of the user 306, the presentdisclosure is not limited to such configurations. Rather, in otherexamples, the display device 304 can determine one or more hapticeffects based one or more events (e.g., an interaction, action,collision, or other event associated with content provided by thedisplay device 304).

For example, the display device 304 may generate and provide a virtualenvironment that includes one or more virtual objects with which theuser 306 can interact as described above. In some examples, one or moreof the virtual objects may be displayed outside of the field of view ofthe user 306 (e.g., behind the user 306 or in front of the user 306, butoutside of the field of view of the user 306). In this example, thedisplay device 304 can determine one or more haptic effects based on thecontent (e.g., the virtual objects) displayed outside the field of viewof the user 306. As an example, the display device 304 is displaying ascene involving various virtual characters engaged in combat and theuser 306 is looking at a particular virtual character engaged in combat.The display device 304 can output a first haptic effect associated withthe particular virtual character being engaged in combat and the user306 can perceive the first haptic effect. The first haptic effect can bea strong vibration to allow the user 306 to perceive haptic effectsassociated with the particular virtual character that the user 306 islooking at. The display device 304 may also determine and output asecond haptic effect associated with other virtual characters engaged incombat outside the field of view of the user 306 (e.g., virtualcharacters engaged in combat in a portion of the scene behind the user306 or virtual characters engaged in combat in a portion of the scene infront of the user 306 and outside the field of view of the user 306).The second haptic effect can be a weak vibration that can allow the user306 to perceive haptic effects associated with virtual charactersengaged in combat behind the user 306 or virtual characters engaged incombat in front of the user, but outside of the field of view of theuser 306. In this manner, the user 306 may experience haptic effectsassociated with a particular virtual character that the user is lookingat and other virtual characters outside the field of view of the user.

FIG. 4 shows an embodiment of a system for haptic feedback using fieldof view according to another embodiment.

As described above, in some examples, one or more haptic effects can bedetermined based at least in part on sensor data indicating that a useris looking at a portion of a display device (e.g., the display device136 of FIGS. 1 and 2 or the display device 304 of FIG. 3). In theexample depicted in FIG. 4, a display device 400 is shown as beingdivided into a first portion 402 (e.g., a top portion of the displaydevice 400), a second portion 404 (e.g., a left portion of the displaydevice 400), a third portion 406 (e.g., a middle portion of the displaydevice 400), and a fourth portion 408 (e.g., a right portion of thedisplay device 400). In this example, the display device 400 can beconfigured in substantially the same manner as the display device 136 ofFIG. 1 and a processor of the display device 400 can determine that auser (not shown) is looking at the first portion 402, the second portion404, the third portion 406, or the fourth portion 408 based on sensordata (e.g., data from a sensor of the display device 400). The displaydevice 400 can determine one or more haptic effects or haptic tracksassociated with the portion of the display device 304 that the user islooking at.

As an example, the display device 400 can be a head-mounted displaydevice that includes a sensor for detecting a motion of the head of auser of the display device 400 or for detecting the head posture of theuser of the display device 400. The sensor can transmit data about themotion of the user's head or data about the user's head posture to theprocessor of the display device 400 that can determine the direction ofthe field of view of the user based on the sensor data, which can beused to determine whether the user is looking at the first portion 402,the second portion 404, the third portion 406, or the fourth portion408. The display device 400 can then determine one or more hapticeffects or haptic tracks associated with the portion of the displaydevice 400 that the user is looking at and the haptic effects or haptictracks can be output by a haptic output device of the display device 400(e.g., the haptic output device 159 of FIG. 1) or a haptic output deviceof a computing device associated with the user (e.g., the computingdevice 101 a of FIG. 2).

For example, the sensor data indicates that the user is looking at thefirst portion 402 of the display device 400, which includes a sceneinvolving combat between one or more virtual characters. The displaydevice 400 can determine and output, via a haptic output device, ahaptic track (e.g., a series of haptic effects, such as, for example,vibrations) associated with the first portion 402 of the display device400 to allow the user of the display device 400 to experience hapticeffects associated with the scene displayed on the first portion 402 ofthe display device 400. As another example, the sensor data indicatesthat the user is looking at the second portion 404 of the display device400, which includes a scene involving a series of car collisions and thedisplay device 400 can determine and output another haptic trackassociated with the series of car collisions to allow the user toperceive haptic effects associated with the series of car collisions.

In some embodiments, the display device 400 can determine that the useris simultaneously looking at more than one portion of the display device400 and the display device 400 can determine a haptic track or hapticeffect associated with each portion of the display device 400 or acharacteristic of each haptic track or haptic effect based on thisdetermination. As an example, the user is looking at the third portion406 of the display device 400, which includes a scene involving avirtual character engaged in combat. The user is also simultaneouslylooking at the fourth portion 408 of the display device 400, whichincludes a scene involving a car rewing its engine. The display device400 can determine and output one or more haptic tracks based on the usersimultaneously looking at both the third portion 406 and the fourthportion 408 of the display device 400.

For example, the display device 400 can determine a first haptic trackassociated with the third portion 406 (e.g., a series of vibrationscorresponding to the virtual character being engaged in combat). Thedisplay device 400 can also determine a second haptic track associatedwith the fourth portion 408 (e.g., a series of vibrations correspondingto the car rewing its engine). The display device 400 can output both ofthe first and second haptic tracks to the user via a haptic outputdevice. In some embodiments, a single haptic track may comprise thefirst and second haptic tracks. In some examples, the display device 400can adjust a characteristic of the first or second haptic track based ona field of view or direction of an eye gaze of the user. For example,the user's head posture may not change, but the field of view ordirection of the eye gaze of the user may shift such that the user ismore focused on the fourth portion 408 than the third portion 406. Insuch examples, the display device 400 can adjust a characteristic of thefirst or second haptic track based on the field of view or direction ofthe eye gaze of the user. As an example, the display device 400 canincrease a magnitude of the second haptic track in response todetermining that the field of view or direction of the eye gaze of theuser is directed more toward the fourth portion 408 than the thirdportion 406. In this example, the display device 400 may also decrease amagnitude of the first haptic track in response to determining that thefield of view or direction of the eye gaze of the user is directed moretoward the fourth portion than the third portion 406. In this manner,the user of the display device 400 may experience haptic effectsassociated with both the third portion 406 and fourth portion 408 of thedisplay device 400 that the user is looking at.

In some embodiments, the display device 400 can be configured to outputa single haptic track associated with the portions 402, 404, 406, 408.The haptic track can include a haptic effect (e.g., a vibration) or aseries of haptic effects that correspond to content (e.g., a video)being displayed on the display device 400 (e.g., via portions 402, 404,406, 408). As an example, the display device 400 is displaying a videothat includes a series of car collisions and various explosions. Thehaptic track can include one or more vibrations that correspond to thecar collisions or explosions.

In some examples, the sensor of the display device 400 can detect amotion of the user's head or detect the user's head posture and thedisplay device 400 can determine a characteristic of the haptic track(e.g., determine a magnitude of one or more vibrations in the haptictrack) based at least in part on the motion of the user's head or theuser's head posture. As an example, the user's head posture may bepositioned such that the user is looking at or toward the second portion404. The user's head posture may subsequently shift such that the useris looking at or toward the third portion 406. In such examples, thedisplay device 400 can adjust a characteristic of the haptic track basedon the change in the user's head posture. For example, the displaydevice 400 can decrease a magnitude of a haptic effect in the haptictrack that is associated with the second portion 404 (e.g., a vibrationassociated with an explosion displayed on the second portion 404) andincrease a magnitude of another haptic effect in the haptic track thatis associated with the third portion 406 (e.g., a vibration associatedwith a virtual character engaged in combat on the third portion 406) asthe user's head posture changes from being directed toward the secondportion 404 to being directed toward the third portion 406. In thismanner, the display device 400 can output a single haptic track andmodify haptic effects included in the haptic track based on the motionof the user's head or the user's head posture such that the user mayperceive the haptic track as being multiple haptic tracks. While in thisexample, the display device 400 may determine a characteristic (e.g.,magnitude) of a haptic track based at least in part on the motion of theuser's head or the user's head posture, the present disclosure is notlimited to such configurations. Rather, in other examples, the displaydevice may determine a characteristic of a haptic track based at leastin part on a direction of a field of view or eye gaze of the user insubstantially the same manner as described above. In some examples,modifying a haptic effect included in a single haptic track as describedabove can simulate a spatial multi-haptic track using a single haptictrack.

In some examples, the display device 400 can include or becommunicatively coupled to one or more audio devices or systems (notshown), which can be configured to generate electronic signals that willdrive a speaker to output corresponding sounds. The audio device orsystem can be configured to output audio or sounds based on the motionof the user's head or the user's head posture, the direction of a fieldof view, or the direction of the eye gaze of the user. As an example,the user's head posture may be positioned such that the user is lookingat or toward the second portion 404, which includes a scene depicting acar rewing its engine. The display device 400 may transmit one or moresignals to the audio device to cause the audio device to output a soundcorresponding to the car rewing its engine in response to determiningthat the user is looking at or toward the second portion 404. Thedisplay device 400 may also cause the audio device to amplify anintensity or volume of the sound based on the user looking at or towardthe second portion 404. In some such examples, the display device 400may also determine a characteristic of a haptic effect or haptic trackassociated with the scene depicted on the second portion 404 in responseto determining that the user is looking at or toward the second portion404. For example, the display device 400 can increase a magnitude of avibration that corresponds to the car rewing its engine in response todetermining that the user is looking at or toward the second portion404. In this manner, the display device 400 can provide a spatialmulti-haptic track using a single haptic track as described above, andthe haptic track or haptic effects in the haptic track may correspond tosounds provided by an audio device or system (e.g., a spatial audiosystem).

In some examples, the display device 400 can be configured to displaycontent on one or more portions of the display device 400. As anexample, the display device 400 may only display content on the firstportion 402 and may not display content on the second portion 404, thethird portion 406, or the fourth portion 408. For example, the displaydevice 400 can be a virtual reality headset and the display device 400may only display a scene on the first portion 402. In such examples, thedisplay device 400 can determine that a user is looking at the scenedisplayed on the first portion 402 of the display device 400 and thedisplay device 400 can output a haptic track associated with the firstportion 402 to the user or output a haptic track associated with thescene displayed on the first portion 402 to the user. The user maysubsequently look at, or in the direction of, another portion of thedisplay device 400. For example, the user's head posture may be adjustedsuch that the user is looking at the second portion 404 and is no longerlooking at the first portion 402. In such examples, the display device400 can determine that the user is looking at the second portion 404 andcan output another haptic track associated with the second portion 404or another haptic track associated with content displayed on the secondportion 404 to the user in response to determining that the user islooking at the second portion and no longer looking at the first portion402. In some examples, the display device 400 may also stop displayingcontent on the first portion 402 in response to determining that theuser is looking at the second portion 404 and no longer looking at thefirst portion 402. Thus, in some examples, the display device 400 mayprovide content to the user via a particular portion of the displaydevice 400, which can cause the user to look at or toward the particularportion and perceive haptic effects associated with that particularportion of the display device 400. In another example, the displaydevice 400 may provide content to the user via a particular portion ofthe display device 400 that the user is looking at or toward and theuser may perceive haptic effects associated with the particular portionof the display device 400 that the user is looking at or toward.

While in the example described above, the display device 400 may displaycontent on a particular portion of the display device 400 based on theuser's head posture being directed at the particular portion, thepresent disclosure is not limited to such configurations. Rather, inother examples, the display device may display content via one or moreportions of the display device 400 based on a field of view or directionof an eye gaze of a user of the display device 400 (e.g., insubstantially the same manner as described above). Further, while in theexample described above, the display device 400 may only display contenton one portion of the display device 400, the present disclosure is notlimited to such configurations. Rather, in other examples, the displaydevice 400 may display content via one or more portions of the displaydevice 400 individually or simultaneously.

Illustrative Methods for Haptic Feedback for Providing Haptic FeedbackUsing a Field of View

FIG. 5 is a flow chart of steps for performing a method 500 forproviding haptic feedback using a field of view according to oneembodiment.

In some embodiments, the steps in FIG. 5 may be implemented in programcode that is executable by a processor, for example, the processor in ageneral purpose computer, a mobile device, or a server. In someembodiments, these steps may be implemented by a group of processors. Insome embodiments, one or more steps shown in FIG. 5 may be omitted orperformed in a different order. Similarly, in some embodiments,additional steps not shown in FIG. 5 may also be performed. The stepsbelow are described with reference to components described above withregard to the systems shown in FIGS. 1 and 2.

The method 500 begins at step 502 when the sensor 203 of the displaydevice 136 detects a field of view 210 of a user 202 a of a computingdevice 101 relative to a display device 136. In some embodiments, thesensor 203 comprises a camera, a sensor for detecting a head motion orhead posture of the user 202 a, or other suitable device, that candetect the field of view 210 or a direction of an eye gaze 212 a-b ofthe user 202 a or provide data for determining the field of view 210 orthe direction of the eye gaze 212 a-b of the user 202 a.

The method 500 continues at step 504 when a signal about the field ofview 210 or the direction of an eye gaze 212 a-b of the user 202 a istransmitted to a processor 138 of the display device 136 or a processor102 of the computing device 101. In some embodiments, the sensor 203transmits the signal about the field of view 210 or the direction of aneye gaze 212 a-b of the user 202 a to the processor 138 or 102.

The method 500 continues at step 506 when the processor 102 or 138determines that content displayed on the display device 136 and withinthe field of view 210 of the user 202 a is associated with a hapticeffect. In some embodiments, the haptic effect determination module 126of the computing device 101 causes the processor 102 to determine thatcontent displayed on the display device 136 and within the field of view210 of the user 202 a is associated with a haptic effect. In otherembodiments, the haptic effect determination module 162 of the displaydevice 136 causes the processor 138 to determine that content displayedon the display device 136 and within the field of view 210 of the user202 a is associated with a haptic effect.

For example, the processor 102 or 138 can determine that the user 202 ais looking at, or in the direction of, content (e.g., texts, images,sounds, videos, characters, virtual objects, virtual animations, etc.)provided via the display device 136 based on the field of view 210 orthe direction of an eye gaze 212 a-b of the user 202 a. As an example,the sensor 203 can include a camera for monitoring movements of an eyeof the user 202 a or muscles near the eye of the user 202 a and theprocessor 102 or 138 can determine the field of view 210 or thedirection of the eye gaze 212 a-b based on the monitored movements. Asanother example, the sensor 203 monitors or measures electrical activityof the muscles moving the eye of the user 202 a and the processor 102 or138 can determine the field of view 210 or the direction of the eye gaze212 a-b based on the electrical activity of the muscles. The processor102 or 138 can determine that the user 202 a is looking at, or in thedirection of, the content on the display device 136 based on the fieldof view 210 or the direction of the eye gaze 212 a-b of the user 202 a.Based on this determination, the processor 102 or 138 can determine thatcontent displayed on the display device 136 and within the field of view210 of the user 202 a is associated with a haptic effect.

As an example, the processor 138 can determine that virtual objects 204,206 displayed on display device 136 are within the field of view 210 ofthe user 202 a. The processor 138 can access one or more lookup tablesor databases that include data corresponding to various haptic effectsassociated with various content that can be displayed by the displaydevice 136. The processor 138 can access the one or more lookup tablesor databases and determine that virtual objects 204, 206 within thefield of view 210 of the user 202 a are associated with one or morehaptic effects.

The method 500 continues at step 508 when the processor 102 or 138determines the haptic effect associated with the content. In someembodiments, the haptic effect determination module 126 of the computingdevice 101 causes the processor 102 to determine the haptic effectassociated with the content. In other embodiments, the haptic effectdetermination module 162 of the display device 136 causes the processor138 to determine the haptic effect associated with the content.

For example, the processor 138 can determine that content (e.g., virtualobject 204) displayed on the display device 136 is within the field ofview 210 of the user 202 a and is associated with a haptic effect (e.g.,in step 506). Based on this determination, the processor 138 candetermine or select the haptic effect associated with the content (e.g.,the haptic effect associated with virtual object 204). As an example,the processor 138 can access the one or more lookup tables or databasesand select the haptic effect associated with the content.

In some embodiments, the haptic effect can include one or more hapticeffects that can allow the user 202 a to perceive or experience hapticeffects that are relevant to the content that the user 202 a is lookingat. As an example, the user 202 a may be looking at the virtual object204, which can be a virtual character engaged in combat. The hapticeffect can include one or more vibrations to allow the user 202 a toperceive the combat.

In some embodiments, in step 508, the processor 102 or 138 can determineone or more haptic effects based on the user 202 a looking at, or in thedirection of, various content displayed on display device 136. As anexample, the processor 138 can determine that the direction of the fieldof view 210 or the direction of the eye gaze 212 a of the user 202 acorresponds with a position or location of virtual object 204 displayedby the display device 136. The processor 138 can also determine that thedirection of the field of view 210 or the direction of the eye gaze 212b of the user 202 a corresponds with a position or location of anothervirtual object 206 displayed by the display device 136. Based on thisdetermination, the processor 138 can access the one or more lookuptables or databases and determine a first haptic effect associated withthe virtual object 204 and a second haptic effect associated with thevirtual object 206.

The method 500 continues at step 510 when the processor 102 or 138determines a distance between the user 202 a and the display device 136.In some embodiments, the sensor 203 can be a Bluetooth device or othernetwork device configured to detect a location of another Bluetoothdevice by analyzing signal strength between the sensor 203 and theBluetooth device. In some embodiments, the sensor 203 may detect adistance between the sensor 203 or the display device 136 and acomputing device 101 or the user 202 a of the computing device 101(e.g., based on the strength of the Bluetooth signal between the sensor203 and the computing device 101). The sensor 203 can transmit sensorsignals to the processor 102 or 138, which can determine the distancebetween the sensor 203 or the display device 136 and the computingdevice 101 or the user 202 a of the computing device 101 based on thesensor signals. In some embodiments, the sensor 203 may detect alocation of the computing device 101 or a distance between the sensor203 or the display device 136 and the computing device 101 or the user202 a of the computing device 101 via any suitable method or technique.

The method 500 continues at step 512 when the processor 102 or 138determines a characteristic (e.g., a magnitude, duration, location,type, frequency, etc.) of the haptic effect based at least in part onthe distance between the sensor 203 or the display device 136 and thecomputing device 101 or the user 202 a of the computing device 101. Insome embodiments, the haptic effect determination module 126 of thecomputing device 101 causes the processor 102 to determine thecharacteristic of the haptic effect based at least in part on thedistance between the sensor 203 or the display device 136 and thecomputing device 101 or the user 202 a of the computing device 101. Inother embodiments, the haptic effect determination module 162 of thedisplay device 136 causes the processor 138 to determine thecharacteristic of the haptic effect based at least in part on thedistance between the sensor 203 or the display device 136 and thecomputing device 101 or the user 202 a of the computing device 101.

For example, the processor 138 can determine that the user 202 a is neardisplay device 136 or virtual object 204 displayed on display device136. The processor 138 can determine a magnitude of the haptic effectassociated with the virtual object 204 based on the user 202 a beingnear the display device 136 or virtual object 204. As an example, thehaptic effect associated with the virtual object 204 can be a strong orlong haptic effect if the user 202 a is near the display device 136. Asanother example, the processor 138 can determine that the user is farfrom the display device 136 or virtual object 204 displayed on thedisplay device 136 and determine a weak or short haptic effectassociated with the virtual object 204 based on this determination.

The method 500 continues at step 514 when the processor 102 or 138transmits a haptic signal associated with the haptic effect to a hapticoutput device 118. In some embodiments, the haptic effect generationmodule 128 causes the processor 102 to generate and transmit the hapticsignal to the haptic output device 118. In another embodiment, thehaptic effect generation module 164 causes the processor 138 to generateand transmit the haptic signal to the haptic output device 118.

The method 500 continues at step 516 when haptic output device 118outputs the haptic effect based on the user 202 a looking at, toward, orin the direction of, content (e.g., virtual objects 204, 206) associatedwith the haptic effect and displayed on display device 136. In someembodiments, the haptic effect may allow the user 202 a to perceive orexperience haptic effects relevant to the content (e.g., virtual objects204, 206) that the user is looking at.

FIG. 6 is a flow chart of steps for performing another method 600 forproviding haptic feedback using a field of view according to oneembodiment. In some embodiments, the steps in FIG. 6 may be implementedin program code that is executable by a processor, for example, theprocessor in a general purpose computer, a mobile device, or a server.In some embodiments, these steps may be implemented by a group ofprocessors. In some embodiments, one or more steps shown in FIG. 6 maybe omitted or performed in a different order. Similarly, in someembodiments, additional steps not shown in FIG. 6 may also be performed.The steps described below are described with reference to componentsdescribed above with regard to the systems shown in FIGS. 1 and 2.

The method 600 begins at step 602 when the sensor 203 of the displaydevice 136 detects a field of view 210 of a user 202 a of a computingdevice 101 relative to a display device 136. In some embodiments, thesensor 203 comprises a camera, a sensor for detecting a head motion orhead posture of the user 202 a, or other suitable device, that candetect the field of view 210 or a direction of an eye gaze 212 a-b ofthe user 202 a relative to the display device 136 or provide data fordetermining the field of view 210 or a direction of an eye gaze 212 a-bof the user 202 a relative to the display device 136.

The method 600 continues at step 604 when a signal about the field ofview 210 or the direction of an eye gaze 212 a-b of the user 202 a istransmitted to a processor 138 of the display device 136 or a processor102 of the computing device 101. In some embodiments, the signal aboutthe field of view 210 or the direction of an eye gaze 212 a-b of theuser 202 a can be transmitted to the processor 138 or the processor 102in substantially the same manner as described above with respect to step504 of FIG. 5.

The method 600 continues at step 606 when the processor 102 or 138determines a first haptic effect associated with a first content withinthe field of view 210 of the user 202 a. In some embodiments, the hapticeffect determination module 126 of the computing device 101 causes theprocessor 102 to determine the first haptic effect associated with thefirst content within the field of view 210 of the user 202 a. In otherembodiments, the haptic effect determination module 162 of the displaydevice 136 causes the processor 138 to determine the first haptic effectassociated with the first content within the field of view 210 of theuser 202 a.

For example, the processor 138 can receive sensor signals from thesensor 203 (e.g., in step 604) and determine the field of view 210 orthe direction of an eye gaze 212 a-b of the user 202 a based on thesensor signals. The processor 138 can determine that the user 202 a islooking at, or in the direction of, the virtual object 204 based on thisdetermination. The processor 138 can access one or more lookup tables ordatabases that include data corresponding to various haptic effectsassociated with various content that can be displayed by the displaydevice 136. The processor 138 can access the one or more lookup tablesor databases and determine the first haptic effect associated with thevirtual object 204 within the field of view 210 of the user 202 a.

The method 600 continues at step 608 when the processor 102 or 138determines a second haptic effect associated with a second contentwithin the field of view 210 of the user 202 a. In some embodiments, thehaptic effect determination module 126 of the computing device 101causes the processor 102 to determine the second haptic effectassociated with the second content within the field of view 210 of theuser 202 a. In other embodiments, the haptic effect determination module162 of the display device 136 causes the processor 138 to determine thesecond haptic effect associated with the second content within the fieldof view 210 of the user 202 a.

For example, the processor 138 can receive sensor signals from thesensor 203 (e.g., in step 604) and determine the field of view 210 orthe direction of an eye gaze 212 a-b of the user 202 a based on thesensor signals. Based on this determination, the processor 138 candetermine that the user 202 a is looking at, or in the direction of, thevirtual object 206. The processor 138 can access one or more lookuptables or databases that include data corresponding to various hapticeffects associated with various content that can be displayed by thedisplay device 136. The processor 138 can access the one or more lookuptables or databases and determine the second haptic effect associatedwith the virtual object 206 within the field of view 210 of the user 202a.

The method 600 continues at step 610 when the processor 102 or 138determines a proportion of the field of view 210 of the user 202 adirected toward the first content or the second content. In someembodiments, the haptic effect determination module 126 of the computingdevice 101 causes the processor 102 to determine the proportion of thefield of view 210 of the user 202 a directed toward the first content orthe second content. In other embodiments, the haptic effectdetermination module 162 of the display device 136 causes the processor138 to determine the proportion of the field of view 210 of the user 202a directed toward the first content or the second content.

As an example, the processor 138 can receive sensor signals from thesensor 203 (e.g., in step 604) and determine that half of the user'sfield of view 210 is directed toward the virtual object 204 or that halfof the user's field of view 210 is directed toward the virtual object206 based on the sensor signals.

The method 600 continues at step 612 when the processor 102 or 138determines a characteristic of the first haptic effect or the secondhaptic effect based at least in part on the proportion of the field ofview 210 of the user 202 a directed toward the first content or thesecond content. In some embodiments, the haptic effect determinationmodule 126 of the computing device 101 causes the processor 102 todetermine the characteristic of the first haptic effect or the secondhaptic effect based at least in part on the proportion of the field ofview 210 of the user 202 a directed toward the first content or thesecond content. In other embodiments, the haptic effect determinationmodule 162 of the display device 136 causes the processor 138 todetermine the characteristic of the first haptic effect or the secondhaptic effect based at least in part on the proportion of the field ofview 210 of the user 202 a directed toward the first content or thesecond content.

As an example, the processor 138 can determine that half of the user'sfield of view 210 is directed toward the virtual object 204 and thathalf of the user's field of view 210 is directed toward the virtualobject 206 (e.g., in step 610). Based on this determination, theprocessor 138 can adjust a first magnitude of the first haptic effectassociated with the virtual object 204 and adjust a second magnitude ofthe second haptic effect associated with the virtual object 206 to halfthe second magnitude. Thus, in some examples, the characteristic of ahaptic effect perceived by the user 202 a may be proportional to, orvary depending on, the user's eye gaze, line-of-sight, or a portion ofthe user's field of view directed toward particular content.

The method 600 continues at step 614 when the processor 102 or 138transmits a first haptic signal associated with the first haptic effectand a second haptic signal associated with the second haptic effect tothe haptic output device 118. In some embodiments, the haptic effectgeneration module 128 causes the processor 102 to generate and transmitthe first and second haptic signals to the haptic output device 118. Inanother embodiment, the haptic effect generation module 164 causes theprocessor 138 to generate and transmit the first and second hapticsignals to the haptic output device 118.

The method 600 continues at step 616 when haptic output device 118outputs the first haptic effect based on the user 202 a looking at,toward, or in the direction of, the first content (e.g., virtual object204) associated with the first haptic effect and outputs the secondhaptic effect based on the user 202 a looking at, toward, or in thedirection of, the second content (e.g., the virtual object 206)associated with the second haptic effect. In some embodiments, the firstor second haptic effect comprises a vibration, a surface deformation, asqueeze, a poke, and/or a puff of a solid, liquid, gas, or plasma.

FIG. 7 is a flow chart of steps for performing another method 600 forproviding haptic feedback using a field of view according to oneembodiment. In some embodiments, the steps in FIG. 7 may be implementedin program code that is executable by a processor, for example, theprocessor in a general purpose computer, a mobile device, or a server.In some embodiments, these steps may be implemented by a group ofprocessors. In some embodiments, one or more steps shown in FIG. 7 maybe omitted or performed in a different order. Similarly, in someembodiments, additional steps not shown in FIG. 7 may also be performed.The steps described below are described with reference to componentsdescribed above with regard to the systems shown in FIGS. 1 and 2.

The method 700 begins at step 702 when the sensor 203 of the displaydevice 136 detects a field of view 210 of a user 202 a of a computingdevice 101 relative to a display device 136. In some embodiments, thesensor 203 comprises a camera, a sensor for detecting a head motion orhead posture of the user 202 a, or other suitable device, that candetect the field of view 210 or a direction of an eye gaze 212 a-b ofthe user 202 a relative to the display device 136 or provide data fordetermining the field of view 210 or a direction of an eye gaze 212 a-bof the user 202 a relative to the display device 136.

The method 700 continues at step 704 when a signal about the field ofview 210 or the direction of an eye gaze 212 a-b of the user 202 a istransmitted to a processor 138 of the display device 136 or a processor102 of the computing device 101. In some embodiments, the signal aboutthe field of view 210 or the direction of an eye gaze 212 a-b of theuser 202 a can be transmitted to the processor 138 or the processor 102in substantially the same manner as described above with regard to step504 of FIG. 5.

The method 700 continues at step 706 when the processor 102 or 138determines a first haptic effect associated with a first portion of thedisplay device 136 within the field of view 210 of the user 202 a. Insome embodiments, the haptic effect determination module 126 of thecomputing device 101 causes the processor 102 to determine the firsthaptic effect associated with the first portion of the display device136 within the field of view 210 of the user 202 a. In otherembodiments, the haptic effect determination module 162 of the displaydevice 136 causes the processor 138 to determine the first haptic effectassociated with the first portion of the display device 136 within thefield of view 210 of the user 202 a.

For example, the processor 138 can receive sensor signals from thesensor 203 (e.g., in step 704) and determine the field of view 210 orthe direction of an eye gaze 212 a-b of the user 202 a based on thesensor signals. The processor 138 can determine that the user 202 a islooking at, or in the direction of, a left portion of the display device136 based on this determination. The processor 138 can access one ormore lookup tables or databases that include data corresponding tovarious haptic effects associated with various portions of, or locationson, the display device 136. The processor 138 can access the one or morelookup tables or databases and determine the haptic effect associatedwith the left portion of the display device 136 that the user 202 a islooking at or toward. In some embodiments, the first haptic effectassociated with the first portion of the display device 136 may allowthe user 202 a to perceive one or more haptic effects relevant to thefirst portion of the display device 136. For example, the left portionof the display device 136 may include a scene depicting a series ofcollisions and the haptic effect associated with the left portion mayinclude one or more vibrations to allow the user 202 a to perceive orexperience the series of collisions.

The method 700 continues at step 708 when the processor 102 or 138determines a second haptic effect associated with a second portion ofthe display device 136 within the field of view 210 of the user 202 a.In some embodiments, the haptic effect determination module 126 of thecomputing device 101 causes the processor 102 to determine the secondhaptic effect associated with the second portion of the display device136 within the field of view 210 of the user 202 a. In otherembodiments, the haptic effect determination module 162 of the displaydevice 136 causes the processor 138 to determine the second hapticeffect associated with the second portion of the display device 136within the field of view 210 of the user 202 a.

For example, the processor 138 can receive sensor signals from thesensor 203 (e.g., in step 704) and determine the field of view 210 orthe direction of an eye gaze 212 a-b of the user 202 a based on thesensor signals. The processor 138 can determine that the user 202 a islooking at, or in the direction of, a top portion of the display device136 based on this determination. For instance, the user 202 a may besimultaneously looking at the left portion (e.g., in step 706) and thetop portion of the display device 136. The processor 138 can access oneor more lookup tables or databases that include data corresponding tovarious haptic effects associated with various portions of, or locationson, the display device 136. The processor 138 can access the one or morelookup tables or databases and determine the haptic effect associatedwith the top portion of the display device 136. In some embodiments, thesecond haptic effect associated with the second portion of the displaydevice 136 may allow the user 202 a to perceive one or more hapticeffects relevant to the second portion of the display device 136. Forexample, the top portion of the display device 136 may include a scenedepicting a car rewing its engine and the haptic effect associated withthe top portion may include one or more vibrations to allow the user 202a to perceive or experience the revving of the engine.

The method 700 continues at step 710 when the processor 102 or 138determines a proportion of the field of view 210 of the user 202 adirected toward the first portion of the display device 136 or thesecond portion of the display device 136. In some embodiments, thehaptic effect determination module 126 of the computing device 101causes the processor 102 to determine the proportion of the field ofview 210 of the user 202 a directed toward the first or second portionof the display device 136. In other embodiments, the haptic effectdetermination module 162 of the display device 136 causes the processor138 to determine the proportion of the field of view 210 of the user 202a directed toward the first or second portion of the display device 136.

As an example, the processor 138 can receive sensor signals from thesensor 203 (e.g., in step 704) and determine that one-fourth of theuser's field of view 210 is directed toward the left portion of thedisplay device 136 (e.g., the first portion) and that three-fourths ofthe user's field of view 210 is directed toward the top portion of thedisplay device 136 (e.g., the second portion) based on the sensorsignals.

The method 700 continues at step 712 when the processor 102 or 138determines a characteristic of the first haptic effect or the secondhaptic effect based at least in part on the proportion of the field ofview 210 of the user 202 a directed toward the first portion or thesecond portion of the display device 136. In some embodiments, thehaptic effect determination module 126 of the computing device 101causes the processor 102 to determine the characteristic of the firsthaptic effect or the second haptic effect based at least in part on theproportion of the field of view 210 of the user 202 a directed towardthe first portion or the second portion of the display device 136. Inother embodiments, the haptic effect determination module 162 of thedisplay device 136 causes the processor 138 to determine thecharacteristic of the first haptic effect or the second haptic effectbased at least in part on the proportion of the field of view 210 of theuser 202 a directed toward the first portion or the second portion ofthe display device 136.

As an example, the processor 138 can determine that one-fourth of theuser's field of view 210 is directed toward the left portion of thedisplay device 136 and that three-fourths of the user's field of view210 is directed toward the top portion of the display device 136 (e.g.,in step 710). Based on this determination, the processor 138 can adjusta first magnitude of the first haptic effect associated with the leftportion of the display device 136 to one-fourth the first magnitude andadjust a second magnitude of the second haptic effect associated withthe top portion of the display device 136 to three-fourths the secondmagnitude. Thus, in some examples, the characteristic of a haptic effectperceived by the user 202 a may be proportional to, or vary dependingon, the user's eye gaze 212 a-b, line-of-sight, or a portion of theuser's field of view 210 directed toward a particular portion of thedisplay device 136 that is associated with a haptic effect.

The method 700 continues at step 714 when the processor 102 or 138transmits a first haptic signal associated with the first haptic effectand a second haptic signal associated with the second haptic effect tothe haptic output device 118. In some embodiments, the haptic effectgeneration module 128 causes the processor 102 to generate and transmitthe first and second haptic signals to the haptic output device 118. Inanother embodiment, the haptic effect generation module 164 causes theprocessor 138 to generate and transmit the first and second hapticsignals to the haptic output device 118.

The method 700 continues at step 716 when the haptic output device 118outputs the first haptic effect based on the user 202 a looking at,toward, or in the direction of, the first portion of the display device136 (e.g., the left of display device 136) associated with the firsthaptic effect and outputs the second haptic effect based on the user 202a looking at, toward, or in the direction of the second portion of thedisplay device 136 (e.g., the top portion of the display device 136)associated with the second haptic effect. In some embodiments, the firstor second haptic effect comprises a vibration, a surface deformation, asqueeze, a poke, and/or a puff of a solid, liquid, gas, or plasma.

In this manner, the systems for haptic feedback based on a field of viewdescribed herein can provide a user with a more immersive experience asthe user views content on a display device by providing one or morehaptic effects relevant to specific content that the user is looking ator relevant to a particular portion of a display device that the user islooking at. Thus, for instance, the user can perceive haptic effectsrelative to the content that the user is looking at so that the user isnot overwhelmed with haptic effects associated with content that theuser is not looking at or toward.

General Considerations

The methods, systems, and devices discussed above are examples. Variousconfigurations may omit, substitute, or add various procedures orcomponents as appropriate. For instance, in alternative configurations,the methods may be performed in an order different from that described,and/or various stages may be added, omitted, and/or combined. Also,features described with respect to certain configurations may becombined in various other configurations. Different aspects and elementsof the configurations may be combined in a similar manner. Also,technology evolves and, thus, many of the elements are examples and donot limit the scope of the disclosure or claims.

Specific details are given in the description to provide a thoroughunderstanding of example configurations (including implementations).However, configurations may be practiced without these specific details.For example, well-known circuits, processes, algorithms, structures, andtechniques have been shown without unnecessary detail in order to avoidobscuring the configurations. This description provides exampleconfigurations only, and does not limit the scope, applicability, orconfigurations of the claims. Rather, the preceding description of theconfigurations will provide those skilled in the art with an enablingdescription for implementing described techniques. Various changes maybe made in the function and arrangement of elements without departingfrom the spirit or scope of the disclosure.

Also, configurations may be described as a process that is depicted as aflow diagram or block diagram. Although each may describe the operationsas a sequential process, many of the operations can be performed inparallel or concurrently. In addition, the order of the operations maybe rearranged. A process may have additional steps not included in thefigure. Furthermore, examples of the methods may be implemented byhardware, software, firmware, middleware, microcode, hardwaredescription languages, or any combination thereof. When implemented insoftware, firmware, middleware, or microcode, the program code or codesegments to perform the necessary tasks may be stored in anon-transitory computer-readable medium such as a storage medium.Processors may perform the described tasks.

Having described several example configurations, various modifications,alternative constructions, and equivalents may be used without departingfrom the spirit of the disclosure. For example, the above elements maybe components of a larger system, wherein other rules may takeprecedence over or otherwise modify the application of the invention.Also, a number of steps may be undertaken before, during, or after theabove elements are considered. Accordingly, the above description doesnot bound the scope of the claims.

The use of “adapted to” or “configured to” herein is meant as open andinclusive language that does not foreclose devices adapted to orconfigured to perform additional tasks or steps. Additionally, the useof “based on” is meant to be open and inclusive, in that a process,step, calculation, or other action “based on” one or more recitedconditions or values may, in practice, be based on additional conditionsor values beyond those recited. Headings, lists, and numbering includedherein are for ease of explanation only and are not meant to belimiting.

Embodiments in accordance with aspects of the present subject matter canbe implemented in digital electronic circuitry, in computer hardware,firmware, software, or in combinations of the preceding. In oneembodiment, a computer may comprise a processor or processors. Theprocessor comprises or has access to a computer-readable medium, such asa random access memory (RAM) coupled to the processor. The processorexecutes computer-executable program instructions stored in memory, suchas executing one or more computer programs including a sensor samplingroutine, selection routines, and other routines to perform the methodsdescribed above.

Such processors may comprise a microprocessor, a digital signalprocessor (DSP), an application-specific integrated circuit (ASIC),field programmable gate arrays (FPGAs), and state machines. Suchprocessors may further comprise programmable electronic devices such asPLCs, programmable interrupt controllers (PICs), programmable logicdevices (PLDs), programmable read-only memories (PROMs), electronicallyprogrammable read-only memories (EPROMs or EEPROMs), or other similardevices.

Such processors may comprise, or may be in communication with, media,for example tangible computer-readable media, that may storeinstructions that, when executed by the processor, can cause theprocessor to perform the steps described herein as carried out, orassisted, by a processor. Embodiments of computer-readable media maycomprise, but are not limited to, all electronic, optical, magnetic, orother storage devices capable of providing a processor, such as theprocessor in a web server, with computer-readable instructions. Otherexamples of media comprise, but are not limited to, a floppy disk,CD-ROM, magnetic disk, memory chip, ROM, RAM, ASIC, configuredprocessor, all optical media, all magnetic tape or other magnetic media,or any other medium from which a computer processor can read. Also,various other devices may comprise computer-readable media, such as arouter, private or public network, or other transmission device. Theprocessor, and the processing, described may be in one or morestructures, and may be dispersed through one or more structures. Theprocessor may comprise code for carrying out one or more of the methods(or parts of methods) described herein.

While the present subject matter has been described in detail withrespect to specific embodiments thereof, it will be appreciated thatthose skilled in the art, upon attaining an understanding of theforegoing may readily produce alterations to, variations of, andequivalents to such embodiments. Accordingly, it should be understoodthat the present disclosure has been presented for purposes of examplerather than limitation, and does not preclude inclusion of suchmodifications, variations and/or additions to the present subject matteras would be readily apparent to one of ordinary skill in the art.

What is claimed is:
 1. A system comprising: a display; a sensorconfigured to detect a field of view of a user relative to the displayand transmit a sensor signal associated with the field of view; aprocessor communicatively coupled to the sensor and configured to:determine that content on the display is within the field of view of theuser based at least in part on the sensor signal; determine a virtualdistance between the user and the content; determine a proportion of thefield of view directed to the content; determine a haptic effect basedat least in part on the virtual distance and the proportion of the fieldof view directed to the content; and transmit a haptic signal associatedwith the haptic effect to a haptic output device configured to outputthe haptic effect.
 2. The system of claim 1, wherein the processor isfurther configured to determine one or more of: a magnitude, duration,location, type, or frequency of the haptic effect based in part on thevirtual distance.
 3. The system of claim 1, wherein the sensor isfurther configured to detect a physical distance between the user andthe display, and wherein the processor is further configured todetermine the haptic effect based at least in part on the physicaldistance.
 4. The system of claim 1, wherein the processor is furtherconfigured to access at least one of a lookup table or a databasecomprising data corresponding to at least one parameter of the contentand to determine the virtual distance based at least in part on the atleast one parameter of the content.
 5. The system of claim 4, whereinthe at least one parameter of the content comprises at least one of asize of the content, a location on the display of the content, or anangle of display of the content.
 6. The system of claim 1, wherein theprocessor is further configured to adjust the haptic effect based on achange of the virtual distance.
 7. The system of claim 1, wherein theprocessor is further configured to: determine a second haptic effectassociated with second content within the field of view of the user, thesecond haptic effect being based at least in part on a second virtualdistance between the user and the second content; and transmit a secondhaptic signal associated with the second haptic effect to the hapticoutput device.
 8. The system of claim 7, wherein the processor isfurther configured to determine a characteristic of the haptic effectand the second haptic effect based at least in part on the proportion ofthe field of view directed toward the content or the second content. 9.The system of claim 7, wherein the processor is further configured to:determine a first eye-gaze direction of the user directed toward thecontent; determine a second eye-gaze direction of the user directedtoward the second content; and determine a characteristic of the hapticeffect or the second haptic effect based at least in part on a changefrom the first eye-gaze direction and the second eye-gaze direction. 10.The system of claim 1, wherein the display is coupled to a virtualreality headset and the processor is further configured to: generate avirtual reality environment; and transmit data that includes the virtualreality environment and the content to the display, the contentincluding a plurality of virtual objects.
 11. A method comprising:detecting a field of view of a user relative to a display; determiningthat content on the display is within the field of view of the user;determining a virtual distance between the user and the content;determining a proportion of the field of view directed to the content;determining a haptic effect based at least in part on the virtualdistance and the proportion of the field of view directed to thecontent; and transmitting a haptic signal associated with the hapticeffect to a haptic output device configured to output the haptic effect.12. The method of claim 11, further comprising determining one or moreof: a magnitude, duration, location, type, or frequency of the hapticeffect based in part on the virtual distance.
 13. The method of claim11, further comprising: detecting a physical distance between the userand the display; and determining the haptic effect based at least inpart on the physical distance.
 14. The method of claim 11, furthercomprising: accessing at least one of a lookup table or a databasecomprising data corresponding to at least one parameter of the content;and determining the virtual distance based at least in part on the atleast one parameter of the content.
 15. The method of claim 14, whereinthe at least one parameter of the content comprises at least one of asize of the content, a location on the display of the content, or anangle of display of the content.
 16. The method of claim 11, furthercomprising adjusting the haptic effect based on a change of the virtualdistance.
 17. The method of claim 11, further comprising: determining asecond haptic effect associated with second content within the field ofview of the user, the second haptic effect being based at least in parton a second virtual distance between the user and the second content;and transmitting a second haptic signal associated with the secondhaptic effect to the haptic output device.
 18. The method of claim 17,further comprising determining a characteristic of the haptic effect andthe second haptic effect based at least in part on the proportion of thefield of view of the user directed toward at least one of the content orthe second content.
 19. The method of claim 17, further comprising:determining a first eye-gaze direction of the user directed toward thecontent; determine a second eye-gaze direction of the user directedtoward the second content; and determine a characteristic of the hapticeffect or the second haptic effect based at least in part on a changefrom the first eye-gaze direction and the second eye-gaze direction. 20.A non-transitory computer-readable medium comprising program code, whichwhen executed by a processor is configured to cause the processor to:detect a field of view of a user relative to a display; determine thatcontent on the display is within the field of view of the user;determine a virtual distance between the user and the content; determinea proportion of the field of view directed to the content; determine ahaptic effect based at least in part on the virtual distance and theproportion of the field of view directed to the content; and transmit ahaptic signal associated with the haptic effect to a haptic outputdevice configured to output the haptic effect.